Millions of households are using autonomous robot cleaners every day. The latest generations of vacuum cleaners offer smart features, such as the possibility to use drops of water to clean floors before hoovering. Such advanced features require a high degree of environmental awareness, including the need to recognize the type of floor, so the robot can change cleaning modes and, for example, avoid spraying water on carpet floors. Using artificial intelligence in robot vacuum cleaners makes these devices more environmentally aware.

Approach

The type of floor can be identified based on its level of softness. If the robot vacuum cleaner detects soft material, like a carpet, it can change the cleaning mode to avoid using water on this type of surface.
  • In this project we used the signal data from the VL53L5 Time-of-Flight (ToF) sensor with 8 x 8 multi-zone detection, which was integrated in the front of a robot cleaner (4.5cm above the floor and 21.5 degrees tilted).
  • Then, we created a collection of different types of material (including soft and hard floor materials) and trained the neural network (NN) model before pre-processing and post-processing the information to improve accuracy.
  • Finally, we implemented the NN model into an MCU, using a NUCLEO-F401RE board, thanks to the STM32Cube.AI software package.

In comparison with standard programming, AI algorithms offer higher levels of accuracy and can easily adapt to special use cases.

Sensor

Time-of-Flight (reference: VL53L5CX)

Data

Dataset signal rate from ToF (output: hard or soft)
Data format 8x8 range @15Hz

Results

Model Multilayer Perceptron (MLP)
Memory footprint:
68 Kbytes of flash memory for weights
1.6 Kbyteof RAM for activations
Accuracy: 96% on more than 50 pieces of material around 200,000 samples
Performance on STM32F401 @84MHz
Inference time: 7 ms

use-case-stm32-cube-ai-confusion-matrix-floor-type-detection use-case-stm32-cube-ai-confusion-matrix-floor-type-detection use-case-stm32-cube-ai-confusion-matrix-floor-type-detection

Confusion matrix

Optimized with
STM32Cube.AI
STM32Cube.AI
Compatible with
STM32 series
STM32 series

Resources

Optimized with STM32Cube.AI

A free STM32Cube expansion package, X-CUBE-AI allows developers to convert pretrained AI algorithms automatically, such as neural network and machine learning models, into optimized C code for STM32.

STM32Cube.AI STM32Cube.AI STM32Cube.AI

Compatible with STM32 series

The STM32 family of 32-bit microcontrollers based on the Arm Cortex®-M processor is designed to offer new degrees of freedom to MCU users. It offers products combining very high performance, real-time capabilities, digital signal processing, low-power / low-voltage operation, and connectivity, while maintaining full integration and ease of development.

STM32 series STM32 series STM32 series
You might also be interested by

Vision | STM32Cube.AI | STM32 AI MCU | Partner | Video | Smart home | Smart building

How to personalize smart home with familiar face identification

embedUR's on-device face authentication embedded on STM32N6 with easy mobile enrollment

Vision | STM32Cube.AI | STM32 AI MCU | Partner | Smart home | Wearables | Microphone | Accelerometer | Tutorial

Handheld development platform for real-time vision, motion, and voice at the edge

All‑in‑one STM32N6‑based platform for on‑device edge AI with NPU acceleration

Industrial | Smart city | Vision | STM32Cube.AI | STM32 AI MCU | Partner | Video | Smart home

Secure entry systems using id3 face recognition with liveness detection

Edge processing with RGB camera and ToF sensor ensures rapid and secure anti-spoofing for access control using the STM32N6 MCU.