Learn / IoT and AI at the edge / Dashboards and logging

Reliable Telemetry & Offline Recovery

ReviewedAdvanced ProAbout 5 min
23.4°CSSD1306
In short

Reliable Telemetry & Offline Recovery: the short answer

Networks drop out. A connected device should keep measuring, store readings for a while, and send them when the connection returns, without losing order or sending the same reading twice as two different events.

Level
Advanced
Time
About 5 min
Topic
IoT and AI at the edge
You will learn to

Keep data safe through network outages with timestamps, a bounded queue and status messages.

Before this: Connected Embedded Devices, Measurement Logging & Calibration

Networks drop out. A connected device should keep measuring, store readings for a while, and send them when the connection returns, without losing order or sending the same reading twice as two different events.

Give every reading a timestamp from the time it was measured, not when it was sent. Get the time from NTP when Wi-Fi is up, and count from the last good time when it is not.

Use a bounded queue: a fixed number of slots in memory or on an SD card. When it is full, decide on purpose what to drop, usually the oldest. Never let a backlog crash the device.

Send a status with the data: sensor OK or failed, Wi-Fi signal, uptime and firmware version. Use MQTT's retained 'last will' message to mark the device offline if it disconnects unexpectedly.

Assume the network will fail

A field device loses its connection for minutes or days. Design for that: store each reading with its own timestamp in a local buffer, send what you can, and keep the rest until the server confirms receipt. Delete a reading only after that confirmation, not after the send.

A buffer in RAM is lost on a reset. For readings that matter, write them to flash or an SD card, and limit how often you write, since flash wears out.

Make resending safe

If a send succeeds but the confirmation is lost, the device sends the reading again. Give every reading a unique id or a sequence number, so the server can drop duplicates. When the link returns, send the backlog in order and slowly enough that you do not flood the server or drain the battery.

When the buffer fills, decide what to drop: usually the oldest readings, or thin out the old ones by keeping every tenth one. Make the choice on purpose and report that data was lost.

Time and health

A device without a real-time clock has wrong timestamps after a reset until it syncs time. Mark readings taken before a sync, or keep a counter of seconds since boot. Send a regular health message with battery level, signal strength and free memory, so a silent device can be told apart from a healthy one that simply had nothing to report.

Common mistakes
  • Deleting readings when they are sent instead of when they are confirmed.
  • Resending without ids, so the server stores duplicates.
  • Keeping the buffer only in RAM, so a reset loses the data.
Advanced lesson · Pro plan

Practise this lesson with Pro

The explanation above is free for everyone. The practice for Advanced lessons in every domain are part of the Pro plan at ₹999 a month.

With Pro you also get: a worked example, a hands-on task, a 3-question check.

Common questions

What will I learn in "Reliable Telemetry & Offline Recovery"?

Keep data safe through network outages with timestamps, a bounded queue and status messages.