Automating Ergonomics: Scalable AI for Technical Hand Grip Classification

Recent studies show that most Work-Related Musculoskeletal Disorders (WMSDs) of the hand and wrist are the result of repeated stresses, caused by repetitive activities over a long time. Tools like ergonomics risk indexes, have been developed to assess upper limb overload in repetitive tasks: part of this evaluation is based on the observation of the number and frequency of Technical Hand Grips (THGs), differing by the number of fingers involved, the exerted forces and most affected regions. Ergonomists typically assess THGs manually via video review, which is time-consuming: a need arises for the automation of these procedures. The rapid and functional creation of AI models capable of classifying efficiently THGs presents some particular challenges: (1) labeling a huge amount of images at ergonomist-level, since, to the best of our knowledge, there are no useful datasets available; (2) considering object occlusions as an intrinsic requirement of realistic study of THGs; (3) moving towards the shared request of data anonymization. This work proposes an Active Learning system in which an initial model labels THGs from videos and consults an Oracle (ergonomist) only when uncertain. An implemented AI (GNN) model is trained on these carefully labeled data to replace the first rudimentary model. The process repeats until the desired model accuracy or data set size is reached. This has guaranteed: (1) a rapid labeling strategy and scaling of the THGs classification model that learns from selected “good” data; (2) implement an occlusion-robust model, by applying selective masking strategies; (3) anonymization through graph-structured data.