Reliable Telemetry & Offline Recovery: the short answer
Networks drop out. A connected device should keep measuring, store readings for a while, and send them when the connection returns, without losing order or sending the same reading twice as two different events.
- Level
- Advanced
- Time
- About 5 min
- Topic
- IoT and AI at the edge
Keep data safe through network outages with timestamps, a bounded queue and status messages.
Before this: Connected Embedded Devices, Measurement Logging & Calibration
Networks drop out. A connected device should keep measuring, store readings for a while, and send them when the connection returns, without losing order or sending the same reading twice as two different events.
Give every reading a timestamp from the time it was measured, not when it was sent. Get the time from NTP when Wi-Fi is up, and count from the last good time when it is not.
Use a bounded queue: a fixed number of slots in memory or on an SD card. When it is full, decide on purpose what to drop, usually the oldest. Never let a backlog crash the device.
Send a status with the data: sensor OK or failed, Wi-Fi signal, uptime and firmware version. Use MQTT's retained 'last will' message to mark the device offline if it disconnects unexpectedly.
Assume the network will fail
A field device loses its connection for minutes or days. Design for that: store each reading with its own timestamp in a local buffer, send what you can, and keep the rest until the server confirms receipt. Delete a reading only after that confirmation, not after the send.
A buffer in RAM is lost on a reset. For readings that matter, write them to flash or an SD card, and limit how often you write, since flash wears out.
Make resending safe
If a send succeeds but the confirmation is lost, the device sends the reading again. Give every reading a unique id or a sequence number, so the server can drop duplicates. When the link returns, send the backlog in order and slowly enough that you do not flood the server or drain the battery.
When the buffer fills, decide what to drop: usually the oldest readings, or thin out the old ones by keeping every tenth one. Make the choice on purpose and report that data was lost.
Time and health
A device without a real-time clock has wrong timestamps after a reset until it syncs time. Mark readings taken before a sync, or keep a counter of seconds since boot. Send a regular health message with battery level, signal strength and free memory, so a silent device can be told apart from a healthy one that simply had nothing to report.
- Deleting readings when they are sent instead of when they are confirmed.
- Resending without ids, so the server stores duplicates.
- Keeping the buffer only in RAM, so a reset loses the data.
Practise this lesson with Pro
The explanation above is free for everyone. The practice for Advanced lessons in every domain are part of the Pro plan at ₹999 a month.
With Pro you also get: a worked example, a hands-on task, a 3-question check.
