Smart mirrors for fitness: pose estimation and multi-person tracking
- Real-Time Analysis Immediate feedback on workout quality and posture correction.
- Cost-Effectiveness Eliminates the need for expensive coaching fees.
- 24/7 Availability Provides continuous support and feedback at any time.
- Personalization Tailors workout plans based on user profile and real-time data.
- Autonomy: Operates without the need for a constant connection to the cloud, ensuring privacy, reducing latency, and enhancing reliability.
Application principle
The image of the scene is captured by an image sensor. On one end its processed to be displayed on the LCD screen. On the other end, the image is downscaled to be sent to the neural network.
Approach
We used one of the models available in the STM32 Model zoo. The model is the Yolov8n_pose, provided by our partner Ultralytics. This is the state-of-the-art model for pose estimation application, providing accurate keypoints with high frame rate.
Such model requires high computational capabilities, and the most suitable MCU option is the STM32N6 which embeds a neural processing unit, the Neural-ART accelerator.
To easily optimize and deploy this model on the STM32N6, we used the scripts provided in the STM32 Model zoo and the STM32Cube.AI code optimizer.
These technologies are part of a broader range of Edge AI solutions. Check out the ST Edge AI Suite by clicking here.
Sensor
RGB Image sensor.
In the STM32 Model Zoo, we provide the code example with two different image sensors: Sony IMX335 5Mp RGB (part of the STM32N6570-DK) or ST's VD66GY 1.5 Mp RGB Global shutter (ideal for precise movement capture).
Dataset and model
Dataset Internal (ST-modified Coco 201 Person)
Model Yolov8n_pose (implemented in Pytorch by Ultralytics and quantized in int8 format using tensorflow lite converter). Input size: 256 x 256 x 3
Results
Weights: 3.2 MB
Activations: 1.1 MB
Inference time: 36 ms
Inference per second: 28
Pose mAP50 (mean Average Precision): 51.1 %
The yolov8 is a state of the art model for multi-pose estimation. Different tradeoff can be achieved by using other dataset, model or resolution. Balancing these factors is crucial for optimizing AI models to meet specific performance and hardware resource constraints. Find more models in the STM32 Model zoo.
Easily recreate this use-case with the following resources:
- Step by step tutorial
- Yolov8n_pose pre-trained model
- Application code example
Additional resources:
- How to install the STM32 Model zoo
Author: Vincent RICHARD | Last update: January, 2025
Resources
Optimized with STM32Cube.AI
A free STM32Cube expansion package, X-CUBE-AI allows developers to convert pretrained AI algorithms automatically, such as neural network and machine learning models, into optimized C code for STM32.
Most suitable for STM32N6 Series
The STM32 family of 32-bit microcontrollers based on the Arm Cortex®-M processor is designed to offer new degrees of freedom to MCU users. It offers products combining very high performance, real-time capabilities, digital signal processing, low-power / low-voltage operation, and connectivity, while maintaining full integration and ease of development.