Sensitivity Analysis of Subjective Ergonomic Assessment Tools: Sensitivity Analysis of Subjective Ergonomic Assessment Tools


Slide2: Sensitivity Analysis of Subjective Ergonomic Assessment Tools: Impact of Input Information Accuracy on Output (Final Scores) Generation Claudia P. Escobar Occupational Safety & Ergonomics Program Industrial & Systems Engineering Department


Thesis Committee: Thesis Committee Dr. Jerry Davis, Chairman Dr. Robert Thomas Dr. Saeed Maghsoodloo Dr. Nathan Dorris


Contents: Contents Literature Review Identified Gap Objective Methodology Results Limitations Further Research Self-Reporting Conclusions


Literature Review: Literature Review Risk factors such as force, posture, movement, vibration, etc. are thought to directly increase the risk for musculoskeletal disorders. (Li & Buckle, 1999) The validity of an ergonomic assessment tool depends on the level of accuracy an evaluator can achieve when assigning scores to these factors. (Faragasanu & Kumar, 2002)


Literature Review: Literature Review Key factors to select an appropriate ergonomic assessment tool: Ease to use Training level for evaluator Applicability of the results Economic issues Time constraints Equipment required Work disruption Need for a data analyst Usability (adequacy and validity) (Waters, Putz-Anderson, and Baron, 1997; Waters, Baron, and Kemmlert, 1998)


Literature Review: Literature Review Ergonomic assessment tools can be subjective or objective in nature. Subjective tools are more predisposed to evaluator’s bias. (Faragasanu & Kumar, 2002) Self-reporting provides valuable insight into working conditions, and is a low cost, low risk, cost effective method. (Marley & Kumar, 1996; Woodcock, 1986; Ramsay, 1993; Andrews, Norman, & Wells, 1996; Faragasanu & Kumar, 2002)


Literature Review: Literature Review Self-reporting may be biased and have low validity/reliability in relation to the needs of the assessment. (Jacobs, 1998; Li & Buckle, 1999) The level of subjectivity directly affects the tool’s validity and reliability. The higher the reliability, the greater the strength and confidence. (Faragasanu & Kumar, 2002)


Identified Gap : Identified Gap It is required to provide a tool that ensures validity during the input information collection: By means of offering input variables discriminated in categories easily distinguishable. With values that the observer can compare with those existing in the assessed job, and select without making a mistake in the process.


Identified Gap: Identified Gap Risk factors such as force, frequency and duration can be assessed without major difficulties. Posture-based conditions require subjective estimations that may result in biased and inaccurate classifications. Mistakes could be made more frequently when assessing these stressors.


Identified Gap: Identified Gap This investigation was derived from the detected need of evaluating the levels of accuracy required when collecting information for input posture-based variables.


Identified Gap: Identified Gap There are no studies ranking the importance of input variables when considering validity of outcomes. Only JSI offers one of its input variables as the most critical.


Objective: Objective To determine the effects input posture-based variables have on the final hazard level classification, when using subjective ergonomic assessment tools, by means of sensitivity analysis.


Objective (Specific): Objective (Specific) To detect the non-sensitive, sensitive, and critical input posture-based variables for three subjective ergonomic assessment tools.


Methodology: Methodology Tool pre-selection Selection criteria Tools selected Sensitivity analysis Critical variables identification


Tool Pre-selection: Tool Pre-selection Fifteen tools were pre-selected according to their self-reporting applicability. Described in terms of main objective, input/output information, limitations, validity, reliability, and sensitivity.


Tools Evaluated: Tools Evaluated Revised NIOSH Lifting Equation Rapid Upper Limb Assessment (RULA) Rapid Entire Body Assessment (REBA) Ovako Working Posture Analysis System (OWAS) Posture, Activity, Tools and Handling (PATH) Liberty Mutual Tables for Lifting, Carrying, Pushing and Pulling (Snook Tables) Job Strain Index (JSI) ACGIH TLV for Hand Activity


Tools Evaluated: Tools Evaluated ACGIH for Work-Related Musculoskeletal Disorders Screening Tool for Lifting Rodgers Muscle Fatigue Analysis Borg Scales of Perceived Exertion OSHA Screening Tool – VDT Checklist WISHA Lifting Analysis WISHA Hand-Arm Vibration Analysis WISHA Checklist


Tools Evaluated (REBA): Tools Evaluated (REBA)


Selection Criteria: Selection Criteria Input and output data used: Quantitative Qualitative Type of assessment yielded: Objective Subjective Self-reporting potential Focus of the tool’s variables Posture-based


Tools Selected: Tools Selected Rapid Upper Limb Assessment (RULA) (McAtamney & Corlett, 1993) Rapid Entire Body Assessment (REBA) (Hignett & McAtamney, 2000) Job Strain Index (JSI) (Moore & Garg, 1995)


RULA: RULA


RULA: RULA RULA is one of the most popular ergonomic assessment tools in industry. User-friendly. Only an initial estimation is required. No major calculations needed. Perfectly matches the selection criteria for the study.


REBA: REBA


REBA: REBA REBA follows the same principles as RULA. Used for both static and dynamic postures. User-friendly. Uses tables to compute scores. Perfectly matches the selection criteria.


JSI: JSI


JSI: JSI JSI focuses on hand and wrist conditions. Obtained from the product of the six multipliers. Did not absolutely match the selection criteria. Wide applicable and popular. It has been validated.


Sensitivity Analysis: Sensitivity Analysis Creation of data sets (combinations) Correlation analysis (Pearson’s test) Non-sensitive variable identification Brute force method and simple linear regression Critical variables identification


Creation of Data Sets: Creation of Data Sets Created iterating simultaneously each input variable within its range of values (combination). Final scores and final hazard level classifications identified for each combination. Only posture-based variables included. Modifiers were excluded from the iterations.


Data Set (example): Data Set (example) 4 2 3 2 4 4 3 3 2 5 5 5


Data Sets: Data Sets RULA  10,368 combinations. REBA  2,160 combinations. JSI  7,500 combinations.


Correlation Test: Correlation Test RULA


Correlation Test: Correlation Test REBA


Correlation Test: Correlation Test JSI


Correlation Test (Results): Correlation Test (Results) All variables were found sensitive. Sensitive variable  has any kind of influence in the final hazard level classification.


Sensitivity Analysis: Sensitivity Analysis Brute Force Method (RULA and REBA) Simple Linear Regression Model (JSI)


Brute Force Method: Brute Force Method Simple, straight-forward method. Individual disturbance of discrete inputs while the rest remains constant. Uses a base case (expected values). Applied to RULA’s and REBA’s data sets.


Base Case Calculation: Base Case Calculation Example: RULA’s Wrist Expected value for wrist (rounded) 1*20.54% + 2*46.38% + 3*33.08% = 2


Base Case: Base Case RULA REBA


RULA: RULA Extreme postures Change from 5-6 to 7-higher 45° Change from 3-4 to 5-6


REBA: REBA 20° Change from 4-7 to 8-10


Critical Variable: Critical Variable With its change from a specific value to the next, it produces an increase (or decrease) in the hazard level classification.


Critical Variables: Critical Variables RULA: Upper arm Neck Trunk Legs REBA: Trunk Neck Legs Upper arm Wrist


Critical Variables: Critical Variables REBA is more prone to a linear behavior when disturbing critical variables than RULA.


Ranking: Ranking RULA


Ranking: Ranking RULA: Upper arm Neck Trunk Legs


Ranking: Ranking REBA


Ranking: Ranking REBA: Trunk Upper arm Legs Neck Wrist


Results: Results If the posture is near the base case, only the critical variables will directly change the final hazard classification.


Results - RULA: Results - RULA Upper Arm: Shoulder flexion from 45. Added shoulder raised and/or upper arm abduction. Neck: Neutral posture to >10 flexion.


Results - RULA: Results - RULA Trunk: Flexion change from 20. Legs: Any misclassification.


Results - REBA: Results - REBA Trunk: Change from neutral to >20. Change from 20 extension. Neck: Added twist or tilt-to-side conditions. Legs: Change in knee flexion from 60.


Results - REBA: Results - REBA Upper arm: Added shoulder raised and/or arm abduction/rotation conditions. Wrist: Added wrist twist/deviation conditions.


Simple Linear Regression Model: Simple Linear Regression Model For each variable, a coefficient is computed. The smaller the coefficient, the greater the influence on final scores.


Simple Linear Regression Model: Simple Linear Regression Model JSI = 5.761 IE + 23.04 (DE + EM) + 19.66 HWP + 24.58 SW + 46.08 DD – 184.32 R2 = 54.30%


Ranking: Ranking Intensity of exertion Speed of work Hand/wrist posture Duration of exertion and efforts per minute Duration per day


Analysis of the Results: Analysis of the Results Preliminary/complimentary studies. Conclusions Limitations Future research Self-reporting Final conclusions


Preliminary/Complimentary Studies: Preliminary/Complimentary Studies Grouped variables analysis for RULA and REBA.  additive effects.  180,000 combinations for RULA.  55,000 combinations for REBA. Simple linear regression model with more degrees of freedom.  same R2.


Conclusions: Conclusions It is inaccurate to assume that all input variables are equivalent in influence on outcomes. Focus on RULA and REBA should start with upper arm and trunk posture assessment, respectively. Focus on JSI should start on intensity of exertion estimation.


Conclusions – RULA / REBA: Conclusions – RULA / REBA An increment in final hazard level classification was often found when additional awkward conditions were added. The more awkward the posture was found, the more sensitive to changes the tool was.


Conclusions: Conclusions The greatest proportions of combinations from data sets described jobs with high levels of hazards. It is difficult to find a “safe” job.


Conclusions: Conclusions RULA:  48%  5-6  32%  7-higher  20%  lowest REBA:  44%  4-7  43%  8-10  11%  lowest  2%  highest JSI:  24%  safe  76%  risk Safe jobs! RULA  0.81% REBA  1.3% JSI  24%


Conclusions: Conclusions If a medium or high hazardous job is detected, and improvements are performed, are the tools going to provide information that would help determine if such improvements were adequate?


Conclusions: Conclusions If a company wants to evaluate the working conditions for its workers, and uses RULA, REBA, or JSI, is it ever going to find results reflecting a safe work environment? Job =  Tasks


Conclusions: Conclusions Perhaps, before analyzing the tool’s validity, it would be appropriate to study the tool’s approach. A too conservative approach could eliminate the possibility of detecting minor changes and improvements in working conditions.


Limitations: Limitations More techniques for sensitivity analysis could be used. More tools must be analyzed. Sensitivity analysis applied not only to expected values but also to minimum, maximum, and random working conditions.


Self-Reporting: Self-Reporting The results of the study are useful when trying to evaluate how appropriate self-reporting would be if used during an intervention. Training for self-reporter should target critical variables.


Future Research: Future Research Extend the study to more ergonomic assessment tools. Expand and modify the selection criteria used. Include other working scenarios (extreme and random conditions).


Final Conclusions: Final Conclusions The study provides the best results possible, considering its scope and limitations. Because it is known which variables cause the most impact on hazard level determination, methods to ensure accuracy and validity during their assessment can be successfully developed and implemented.


Final Conclusions: Final Conclusions Levels of training should target the critical variables identified. The results of the study provide a valid and strong reference to focus the subjective component that potentially dominate the hazard level outcome.


Acknowledgements: Acknowledgements Dr. Jerry Davis Dr. Robert Thomas Dr. Saeed Maghsoodloo Dr. Nathan Dorris Michael Gray Family and Friends


Slide72: Questions? T h a n k Y o u !


Sensitivity Analysis of Subjective Ergonomic Assessment Tools: Sensitivity Analysis of Subjective Ergonomic Assessment Tools