Personal protective equipment detection

Construction is among the industries where accidents occur most frequently. It's crucial to ensure that workers consistently wear their personal protective equipment to safeguard their safety, reduce accidents, and adhere to regulatory standards. 
Several solutions can be considered:

• Manual inspections: While they allow direct verification, their effectiveness is limited by human capacity and time. They are also prone to errors and are not scalable.
• CCTV surveillance: This covers larger areas, providing a broader overview. However, it requires manual review of footage, which is time-consuming and raises privacy concerns as individuals may feel constantly monitored.
• Wearable sensors: These offer real-time data, enhancing responsiveness to incidents. However, they are often expensive and may face resistance from workers who are reluctant to wear sensors continuously.

This is where edge AI comes in. It offers real-time detection, cost efficiency, privacy preservation, and scalability compared to other solutions. Quantitative evidence can be mentioned, such as the reduction in accident rates and cost savings, while qualitative feedback from workers validates an improved perception of safety.

This use case is based on a public dataset containing images of workers wearing personal protective equipment (helmets, safety vests, shoes, and gloves). The goal is to detect what personal protective equipment a person is wearing and draw a bounding box around each piece of equipment.

For this project, we used the STM32 model zoo. It provides pre-trained models optimized for STM32 and scripts to retrain and customize your models.

First, we cloned the GitHub repository and set up our project environment. We mainly used the resources from the "Object detection" folder since our project involved detecting personal protective equipment. You might use resources from another folder depending on your use case. Several applications are available in the STM32 model zoo.

After downloading the dataset, we imported the files into the STM32 model zoo in the designated folder (stm32ai-modelzoo/object detection/datasets). Each sample in the dataset consists of an image and a txt file with the position of each piece of equipment in the image.

We then used the configuration file (stm32ai-modelzoo/object detection/src/user_config.yaml) to set up the training parameters for our model. This file allowed us to:

• Select one of the pre-trained models available in the STM32 model zoo: SSD mobilenet v1in our case.
• Define the mode of operation: here chain_tqeb to train, quantize, evaluate and benchmark. *
• Define the classes to detect: Person, Helmet, Protection Jacket, Protection Shoes, Gloves.
• Enter the training, validation, test, and quantization folder path (it is recommended to use a portion of the training set)
• Rescale the images to the model input size: 224x224x3.
• The other parameters can be left as default. However, you can modify them to improve the training of your model.

Then, we just had to execute the python script with the command python stm32ai_main.py to start the training (make sure to be in the stm32ai-modelzoo/object_detection/src folder).

Once the training was completed, we were able to retrieve the results in the experiment_outputs folder (stm32ai-modelzoo/object_detection/src/ experiment_outputs). In this folder, we can find the .h5 train model and its .tflite quantized version, the execution time and memory footprint for each model, and the confusion matrix for each object to be detected.

You will find below a video in which we show all these steps.

* The quantization and benchmark are done through the ST Edge AI Developer Cloud. You will need an ST account to be able to use it.