What Is Edge AI and TinyML?
Machine learning models can be trained to recognise patterns: the sound of a word, a shake or a wave of the hand, a person in a camera frame, or a vibration that means a motor bearing is wearing out. Training needs lots of data and computing power, so it happens on a laptop or in the cloud. Example: Match the device to the data: audio and motion fit microcontrollers, single-camera vision fits a Pi, multi-camera video fits a Jetson.
- Level
- Beginner
- Time
- About 6 min
- Topic
- IoT and AI at the edge
Understand how a small device can recognise sounds, gestures or images, and choose between a microcontroller, a Pi and a Jetson.
Before this: Connected Embedded Devices
Machine learning models can be trained to recognise patterns: the sound of a word, a shake or a wave of the hand, a person in a camera frame, or a vibration that means a motor bearing is wearing out. Training needs lots of data and computing power, so it happens on a laptop or in the cloud. Running a trained model, called inference, needs much less, and can happen on the device itself. That is edge AI. TinyML is edge AI on microcontrollers, with models of a few tens to a few hundred kilobytes.
Running on the device has real advantages. It works without internet, answers in milliseconds, keeps private data such as audio on the device, and saves power and data costs because only results are sent, not raw sensor streams. The trade-off is that models must be small and simple. A microcontroller such as an Arduino Nano 33 BLE Sense or an ESP32-S3 can recognise a few keywords or gestures; a Raspberry Pi can classify images a few times a second; a Jetson can detect many objects in live video.
The usual workflow has five steps: collect data from the real sensor, label it, train a model, test it on data it has never seen, and deploy it to the device. Tools such as Edge Impulse let you do all of this in a browser and export a ready-made Arduino library, so you can get a working gesture or keyword detector in an afternoon. Most of the work is in collecting good, varied data; a model trained on one person's voice in a quiet room will fail in a noisy classroom.
What a model really is
A trained model is a large set of numbers (weights) arranged in layers. Giving it an input, such as a second of audio or a camera frame, produces numbers out, such as a probability for each class: "yes", "no", "noise". Nothing is understood in a human sense. The model has learned which patterns in the data go with which label, and it can only be as good as the examples it saw.
That is why the data matters more than the cleverness of the model. A model trained on 50 clean recordings from one person will often be worse than a simpler one trained on 500 varied recordings.
Making a model small enough
A model that runs on a microcontroller has to fit in a few hundred kilobytes of memory and finish in milliseconds. Two common tricks help. Quantisation stores each weight as an 8-bit integer instead of a 32-bit decimal, shrinking the model about four times with a small loss of accuracy. A smaller network design uses fewer layers and fewer weights, trading some accuracy for speed.
The result is a model that recognises a handful of keywords or gestures, not open-ended speech. Choose a task with a small, clear set of answers.
Measuring whether it works
Keep some data aside that the model never saw during training and test on that. Look at the confusion matrix: which classes does it mix up? A model that is 95 % accurate overall can still fail on the one case that matters, such as a rare fault. Then test it in the real place with the real noise, which is the test that counts.
Decide what to do when the model is unsure. A confidence threshold lets you ignore weak guesses, with a trade-off: a high threshold misses real events, a low one raises false alarms.
When not to use machine learning
If a simple rule works, use the rule. A threshold on a temperature, a limit on vibration or a timer is easier to test, explain and fix than a model. Machine learning earns its place when the pattern is hard to describe in rules, such as the sound of a failing bearing or the shape of a gesture.
Be careful with anything safety-related. A model gives a probability, not a guarantee, so never let it alone decide something dangerous.
- Training and testing on the same data, then trusting the high score.
- Collecting data in one room and deploying in another.
- Choosing machine learning when a simple threshold would do.
Choose hardware for three ideas
- Detect 'on' and 'off' spoken commands: microphone, two words plus noise; a Nano 33 BLE Sense or ESP32-S3 is enough.
- Count people passing a door from a camera: low frame rate is fine; a Raspberry Pi with the AI Camera fits.
- Detect vehicles and read number plates on four cameras: needs a GPU, so a Jetson Orin Nano.
- In each case, prefer the smallest device that meets the speed you need.
Result. Match the device to the data: audio and motion fit microcontrollers, single-camera vision fits a Pi, multi-camera video fits a Jetson.
List three problems around you that edge AI could solve (for example, detecting a pump running dry from its sound). For each, write the sensor, the classes to detect, where training data would come from, and whether a microcontroller, a Pi or a Jetson is needed.
3 quick questions
What is inference?
Which is an advantage of running AI on the device?
What usually takes the most effort in a TinyML project?
