This project aims to improve workplace safety management by developing algorithms for the detection of: Nonuse of protective equipment (PPE) (OBJ1), and Unsafe acts during the pushing and pulling activities (OBJ2). The pipeline of the proposed concept is shown bellow.

Figure 1. Pipeline of the proposed concept.

Concept and underpinning methodologies for achieving the OBJ1

Following the graphical illustration in Figure 1, the outcome of OBJ1 will be a four-step AI-based solution able to check if employees follow the safety recommendations related to the PPE usage. The successive steps will be released as connected modules able to perform:

Human posture estimation, which assumes the detection of an employee in images, and localization of the body parts (legs, head, hands, etc).
Inspection of the body-mounted PPE assumes classification of specific regions around hands, legs and torso, so that the output information is the presence of the corresponding PPE.
Generation of PPE reports assumes the development of a compact framework.

Figure 2. The concept for solving the OBJ1.

Concept and underpinning methodologies for achieving the OBJ2

The hypothesis of OBJ2 is that the assessment of both anatomic/biomechanical aspects (body posture and acting forces) and cognitive state (vigilance, attention, cognitive fatigue, micro sleeps, etc.) could help in preventing unsafe acts that may contribute to musculoskeletal disorders (MSD). Composing parts of the decision support system are illustrated in Figure 5.c, and they include:

Human detection and human pose estimation. In OBJ2, this feature will be used for estimation of 3D positions of human body landmark points.
Push and pull force (PPF) will be measured by using an IoT module which will be able to provide accurate real-time measurement of PPF in three directions
Cognitive state of the employee will be assessed via wearable EEG and EMG sensors. These will help us to detect timepoints when employee’s fatigue occurs
Generation of PPE reports, similarly as in OBJ1, assumes development of features for web-based administration and management of detected unsafe P&P acts.

Figure 3. The concept proposed for solving the OBJ2. The aim is to consider a period when an employee is in interaction with the cart (Figure 3.a) and classify pushing and pulling activities as safe and unsafe. The decision will be made by considering employee 3D pose (extracted from IP cameras) and features extracted from EEG and force sensors (Figure 3. b and Figure 3.c).