Ergonomic risk assessments – a need for reliable

and attractive methods Mikael Forsman

[email protected] Karolinska Institutet

Stockholm, Sweden

Co-workers Reliability in ergonomic risk assessments by observation  https://wearnotch.com/ Ida-Märta Rhén, Kristina Eliasson, Katarina Kjellberg, Xuelong Fan (KI) Peter Palm, Teresia Nyman (Uppsala) Per Lindberg (Gävle)

Practical objective measurements for sedentary time and body postures Liyun Yang, Carl Lind (Royal Institute of Technology)

Camilla Dahlqvist, Gert-Åke Hansson (Lund) Jens Wahlström (Umeå)

Mikael Forsman

2016-08-16

2

Mikael Forsman

2016-08-16

Introduction  Work-related musculoskeletal disorders (MSDs) are still frequent.  In order to decrease the disorders, risks need to be identified and quantified.  Takala and co-workers, 2010, evaluated 30 observational methods, and found a need for further reliability testings.  New electronic components facilitates uncomplicated measurements of biomechanical exposure.

Mikael Forsman

2016-08-16

4

Observation Reliability Project 12 experienced ergonomists were given:  10 video-recorded (2-5 minutes) work tasks supermarket work meat cutting and packing engine assembly cleaning post sorting hairdressing.  data of: the work task length, pause- and restsschedules, weights of handled goods, and the employees’ ratings of discomfort, work demands and control, were given for each task.  max time per work task assessment was 20 minutes.  The assessments were repeated about 2 months. Mikael Forsman

2016-08-16

5

Observation Reliability Project They used 6 observation methods:

1. Occupational Repetitive Actions checklist (OCRA) 2. Quick Exposure Checklist (QEC) 3. Strain Index (SI)

4. Assessment of Repetitive Tasks (ART) 5. Hand Arm Risk-assessment Method (HARM) 6. Repetitive work model by the Swedish Work Environment Authority

Observation Reliability Project They used 6 +1 observation methods: 0. Own experience – no method

1. Occupational Repetitive Actions checklist (OCRA) 2. Quick Exposure Checklist (QEC) 3. Strain Index (SI)

4. Assessment of Repetitive Tasks (ART) 5. Hand Arm Risk-assessment Method (HARM) 6. Repetitive work model by the Swedish Work Environment Authority

Mikael Forsman

2016-08-16

7

Observation Method Example - Quick Exposure Check (QEC)

Mikael Forsman

16 augusti 2016

8

Mikael Forsman

2016-08-16

9

Katarina Kjellberg, Mikael Forsman

2016-08-16

10

Mikael Forsman

16 augusti 2016

11

Method 0 – own assessment  21 ergonomists  The risk was rated into:  green (no risk)  yellow (investigate further)  red (immediate risk)  Ratings were done for 8 specific body regions  neck  right and left shoulders  right and left arms/elbows  right and left wrists/hands  lower back  and for the over-all risk.

Left Left elbow Left wrist shoulder Unpack groceries to shelves

Right shoulder

Neck

High risk Moderate risk Low risk

2 9 10

2 12 7

2 10 9

3 14 4

Right elbow

Right wrist Low back

2 9 10

2 8 11

4 10 7

Total risk

0 9 12

3 11 7

Put nets around roasts High risk Moderate risk Low risk

1

17

12

11

2

8

6

6

17

16

4

8

8

11

8

11

14

4

4

0

1

2

8

5

4

1

0

20 Ratings

Throw small boxes in containers High risk Moderate risk Low risk

1

2

2

4

7

7

7

0

5

9

11

13

14

13

10

12

12

15

11

8

6

3

1

4

2

9

1

Put packs of letters into boxes High risk Moderate risk Low risk

1

0

2

2

3

6

6

0

6

6

5

8

7

11

11

13

9

12

14

16

11

12

7

4

2

12

3

Debone meat High risk Moderate risk Low risk

8

5

7

6

6

14

17

2

15

9

6

10

12

8

6

4

10

5

4

10

4

3

7

1

0

9

1

0

0

0

0

0

0

0

0

0

1

3

9

7

4

9

7

1

3

20

18

12

14

17

12

14

20

18

Engine assembly High risk Moderate risk Low risk

Hair cutting High risk Moderate risk Low risk

4

6

1

0

11

6

11

2

7

16

13

6

8

10

12

7

9

12

1

2

14

13

0

3

3

10

2

0

0

0

0

5

7

6

10

5

5

4

6

7

12

11

10

11

15

16

17

15

14

4

3

5

0

1

Toilet cleaning High risk Moderate risk Low risk

Supermarket cashier work High risk Moderate risk Low risk

2

5

3

7

11

3

5

1

14

16

12

10

10

8

10

12

13

7

3

4

8

4

2

8

4

7

0

Cleaning stairs Red Yellow Green

2

1

4

7

3

6

3

1

4

16

9

10

9

15

13

14

4

14

3

11

7

5

3

2

4

16

3

OCRA over-all risk level  Proportional agreement between ergonomists (39%)  Linearly weighted kappa = 0.41 Video-clip Grocery work

Meat netting

Post sorting

Post handling

Meat Engine deboning assembly

Hairdressing

Toilet cleaning

Cashier work

Stair cleaning

1 2

Ergonomist

3 4

5 6

Risk level

7

5 4 3 2 1

8 9

10 11

Ida-Märta Rhén, Mikael Forsman IEA 2015

2016-08-16

14

Test-retest over-all OCRA risk level  Proportional agreement within ergonomists = 46%  Linearly weighted kappa = 0.51 Video-clip Grocery work

Meat netting

Post sorting

Post handling

Meat Engine deboning assembly

Hairdressing

Toilet cleaning

Cashier work

Stair cleaning

1 2

Ergonomist

3 4 5 6 7 8 9 10 = Test-retest agreement Ida-Märta Rhén, Mikael Forsman IEA 2015

2016-08-16

15

Three experts used the methods – fully according to the manuals Consensus results Method

Bedömningsmetoder Uppackning Näta kött Kasta post Fylla post SWEA AFS ART ART HARM HARM OCRA OCRA SI SI QECupper övre extremitet QEC extremity QECgeneral helkropp QEC

Bena ur

Motor

Hår

Toalett

Kassa

Trappstäd

6

Comparison with a similar study

High risk

5

Moderate risk 4 This study:

Low risk

10 9

3

8 7 6

2

High risk

5

Moderate risk 4

Low risk

1

3 2 1

0

0

SWEA

SWEA

QEC general

QEC general

ART

ART

HARM

HARM

OCRA

OCRA

QEC upper extremity

QEC upper extremity

SI

SI

QEC hand/wrist

QEC hand/wrist

Chiasson M-È, Imbeau D, Aubry K, Delisle A. Comparing the results of eight methods used to evaluate risk factors associated with musculoskeletetal disorders. International Journal of Industrial Ergonomics. 2012;42(5):478-88:

Katarina Kjellberg, Mikael Forsman

2016-08-16

17

Statistics for the inter-observer reliability  The Proportional agreement, Pr(a), of the different ergonomist’s ratings (in percent)  Cohen’s pairwise kappa was computed for all pairs

Pr(a) represents the actual observed proportional agreement Pr(e) represents agreement expected by chance.  Linearly weighted kappa, κw, was computed and averaged in the way suggested by Davies and Fleiss (1982). Davies M, Fleiss JL. 1982. Measuring Agreement for Multinomial Data. Biometrics 38, 1047-1051.

Results

Method QEC

Reliability

Linearly Weighted Kappa

Inter-rater Intra-rater 0.49 0.60 0.44 0.58 0.67 0.76 0.87 0.88 0.55 0.68 0.41 0.61

Validity 0.44 0.44 0.79 0.91 0.47 0.36

Total

0.65

0.70

0.62

Left Hand Right Hand The Highest Risk Hand

0.39 0.48 0.47

0.49 0.51 0.55

0.41 0.41 0.40

Left Side Right Side

0.58 0.65

0.65 0.68

0.60 0.49

OCRA

Total

0.37

0.50

0.43

SWEA

Tot Risk Posture and Movements Sitting / Standing Neck Back Arm/Shoulder Leg

0.21 1.00 0.22 0.16 0.21 0.05

0.30 1.00 0.32 0.44 0.13 0.36

0.38

0.27 0.18 0.32 0.12 0.15 0.16 0.18 0.26 0.31

0.35 0.30 0.38 0.23 0.40 0.20 0.28 0.62 0.41

HARM SI

ART

Landis & Koch (1977): < 0.20 – poor 0.21 – 0.40 – fair 0.41 – 0.60 – moderate 0.61 – 0.80 – substantial 0.81 – 1.00 – almost perfect

Risk level for Back Shoulder Wrist Neck Total (with 4 levels) Total (with 3 levels)

Own assessment Neck Right Shoulder Left Shoulder Left Arm/elbow Right Arm/elbow Right Hand/wrist Left Hand/wrist Lower Back Total Risk

HARM Postures Neck-shoulder

HARM Reliability Postures Neck-shoulder Time in demanding postures: green yellow red 0–10 10–50 >50%

Inter-rater w = 0.18

κ

Intra-rater w = 0.21

κ

Landis & Koch (1977): < 0.20 – poor 0.21 – 0.40 – fair 0.41 – 0.60 – moderate 0.61 – 0.80 – substantial 0.81 – 1.00 – almost perfect

HARM Reliability Postures Arm-wrist Time in demanding postures: green yellow red 0–10 10–50 >50%

Inter-rater w = 0.14

κ

Intra-rater w = 0.25

κ

Landis & Koch (1977): < 0.20 – poor 0.21 – 0.40 – fair 0.41 – 0.60 – moderate 0.61 – 0.80 – substantial 0.81 – 1.00 – almost perfect

Observation Reliability Project 12 experienced ergonomists were given:  10 video-recorded (2-5 minutes) work tasks supermarket work meat cutting and packing engine assembly cleaning post sorting hairdressing.  data of: the work task length, pause- and restsschedules, weights of handled goods, and the employees’ ratings of discomfort, work demands and control, were given for each task.

Mikael Forsman

2016-08-16

23

HARM Total risk score

HARM Total risk index (three risk levels)

Inter-rater Reliability Repetition, Movements and Postures Method

κw

low

high

QEC

0.17 - 0.44

Hand/wrist posture

Hand/wrist repetition

HARM

0.14 - 0.3

Hand/arm posture

Force duration/repetition

SI

0.04 - 0.40

Hand posture

Force repetition

ART

0.17 - 0.45

Hand/wrist posture

Hand/arm repetition

OCRA

0.03 - 0.53

Elbow posture

Repetition

SWEA

0.05-0.22

Leg posture/movements

Neck posture/movements

Left elbow

Neck

Own assessment 0.12 - 0.27 Landis & Koch (1977): < 0.20 – poor 0.21 – 0.40 – fair 0.41 – 0.60 – moderate 0.61 – 0.80 – substantial 0.81 – 1.00 – almost perfect

Reliability – other studies in comparison with Method 0 Statistics presented with Cohen’s kappa (κ), Kendall’s coefficient of concordance (KCC), Intra-class correlation (ICC) and Weighted kappa (κw) . Method

Inter-observer reliability

Method 0 own assessment

Reference

Ice-dance US championship 2015 Short program 9 judges

17 pairs Scores: 14.8 – 56.3

Inter-rater w = 0.83

κ

Madison Chock Evan Bates 2015 U.S. Champions

A DREAM!

Mikael Forsman

2016-08-16

28

A DREAM!  It shall be really easy to make biomechanical risk assessments!  And cheap!  For high and low activity jobs!

 With safe job limit values!  With feedback to individual workers!  Everyone can use the methods and understand the results!

Mikael Forsman

2016-08-16

29

the Smart textiles project      

Long Sleeves 3-5 IMUs ECG Bioimpedance Bluetooth SmartPhone Data Collection

A DREAM!

Mikael Forsman

2016-08-16

31

the Smart textiles project The glove

Katarina Kjellberg, Mikael Forsman

2016-08-16

32

the Smart textiles project The partners

Katarina Kjellberg, Mikael Forsman

2016-08-16

33

Katarina Kjellberg, Mikael Forsman

2016-08-16

34

Mikael Forsman

Manual…

There is also a simple Excel application

alysprogram i Excel

psdel som ill höger). n så att

klinometer tt kort 2 tartas 1 sekund, Figur 5. Inklinometern blinka. som används i Excelapplikationen, fasttejpad använder för att kännas av nollvinkeln) på armen. iduret start och stopp per arbetsuppgift.

Accuracy of a posture measurement system for practitioners

Carl LIND Unit of Ergonomics, KTH, Royal Institute of Technology, Huddinge, SWEDEN

Mikael FORSMAN Institute of Environmental Medicine, Karolinska Institutet, Stockholm, SWEDEN

Results Experiment 1A

Experiment 1B

R2 = 0.9975

R2 = 0.9988

NES 2015 - Carl LIND & Mikael FORSMAN

David Dunstan IEA Key-note

Apps for measurements of sedentary times

Mikael Forsman

2016-08-16

54

Conclusion

ErgoSitMeter

Mikael Forsman

2016-08-16

55

Take Home Messages  Low reliability in observational risk assessments of manual work.  Also differences between different methods.  Practical inexpensive inclinometer and applications in Excel and for IPhones exist that result in data of the quality comparable to that of research methods.  It should be possible to combine observations with “easy measurements”.  We need more research on criteria for acceptable ranges – of different parameters.  Attractive! It is hard to attract ergonomists to start measure instead of doing observations.