Quantising and Deploying TinyML Models: the short answer
A model trained in TensorFlow or PyTorch stores its weights as 32-bit floating-point numbers. Microcontrollers have little memory and often no fast floating-point maths, so models are quantised to 8-bit integers.
- Level
- Advanced
- Time
- About 6 min
- Topic
- IoT and AI at the edge
Shrink a neural network with int8 quantisation, fit it into a microcontroller's memory and measure its real speed and accuracy.
Before this: Train a Gesture Model for a Microcontroller, Accurate ADC Readings: Calibration and Filtering
A model trained in TensorFlow or PyTorch stores its weights as 32-bit floating-point numbers. Microcontrollers have little memory and often no fast floating-point maths, so models are quantised to 8-bit integers. Post-training int8 quantisation maps each tensor's range of values onto −128 to 127 using a scale and zero point, which makes the model about four times smaller and usually several times faster, often losing only a percent or two of accuracy. Quantisation needs a representative dataset of a few hundred real samples so the converter can measure the ranges. If accuracy drops too much, quantisation-aware training simulates int8 during training and recovers most of the loss.
On the device, TensorFlow Lite for Microcontrollers (now distributed as LiteRT for Microcontrollers) runs the model from a C array in flash. You give it a tensor arena, a block of RAM for the activations and working buffers. Size it by starting large, running the model, and reading arena_used_bytes(), then add some margin. Flash holds the weights; RAM holds the arena. An ESP32-S3 with PSRAM fits much bigger models than a classic ESP32, and vendor-optimised kernels (ESP-NN on Espressif chips, CMSIS-NN on Arm Cortex-M) make convolutions several times faster.
Measure on the real hardware. Time one inference with micros() over many runs, and check that it fits your window: a keyword spotter that hops every 250 ms must finish each inference well within that. Check accuracy again on the device with the real sensor, because preprocessing differences (scaling, filtering, sample rate) between training and firmware are the most common cause of models that worked in Python but fail on the board. Keep the preprocessing code identical, or generate it from the same tool.
Why quantise
A trained model stores its weights as 32-bit floating-point numbers. Quantisation converts them to 8-bit integers, which cuts the size to about a quarter and makes the model run faster on a microcontroller with no floating-point unit, at the cost of a small drop in accuracy. Post-training quantisation needs a few hundred representative samples to choose the scaling.
Always compare the quantised model against the original on the same test set, since the loss in accuracy varies from model to model.
Fit in memory
Memory has two parts: flash for the model weights and RAM for the tensor arena, the working space the interpreter uses while running. Check both against the chip's limits, including the memory used by the rest of your program. A model that fits in flash may still not fit in RAM. A smaller input, fewer layers and fewer filters all reduce the use.
Measure the time for one prediction on the real chip, and check it is shorter than the interval between windows of data.
Test on the device
Run the same test samples through the model on the device and compare with the results on the computer. If they differ a lot, check the input scaling and the data order. Then test with live sensor data in the real environment, since the model sees noise and conditions that a dataset may not have included.
- Checking only the model size and forgetting the RAM arena.
- Quantising without comparing accuracy on a test set.
- Feeding the device model differently scaled input than the one trained.
Practise this lesson with Pro
The explanation above is free for everyone. The practice for Advanced lessons in every domain are part of the Pro plan at ₹999 a month.
With Pro you also get: a worked example, a hands-on task, a 3-question check.
