Measurement Logging & Calibration: the short answer
A good log is one you can still trust next month. Each row needs the time, the value with its unit, which device and sensor made it, and whether the reading was valid.
- Level
- Intermediate
- Time
- About 5 min
- Topic
- IoT and AI at the edge
Design a log that records time, units, raw values and errors, and check it before plotting.
A good log is one you can still trust next month. Each row needs the time, the value with its unit, which device and sensor made it, and whether the reading was valid.
CSV is the simplest format: one header row, then one row per reading. It opens in any spreadsheet. Store the raw value as well as the converted one; if you later fix a calibration, you can recalculate.
Averaging reduces noise but not bias. A sensor that always reads 1.5 °C high will still be 1.5 °C high after averaging. To find bias, compare with a trusted reference at a few points and record the offset.
Before drawing graphs, look for gaps, error codes and sudden steps. They usually mean a disconnect or restart, not a real change.
Design the record before you collect data
Decide what questions the data must answer, then choose the columns. For each reading note the time, the sensor or device, the raw value, the converted value with its unit, and a status or error code. Add a field for the firmware version or the calibration used. When you change anything, such as the sensor position, write that down with the date too, in a notes file kept next to the data.
Use a stable time format such as ISO 8601 (2026-11-16T14:05:00+05:30). It sorts correctly, spreadsheets understand it, and it states the time zone.
Where to store it
An SD card is the simplest for a device that is not connected: write one line per reading and flush the file every so often, so a power cut loses a few readings, not the whole file. Flash storage wears out if you write constantly, so for a long log write in batches. On a Pi, log to a file or a small database such as SQLite, and back up the file regularly.
Name files by date so they stay a manageable size, and never overwrite old data on restart.
Sample rate and file size
Sample at least twice as fast as the quickest change you care about, and no faster than you can store. One reading a second makes 86,400 lines a day, about 3 to 4 MB as text. A reading a minute is 1,440 lines, tiny. Use the slowest rate that answers your question, and a higher one only around events you want to see.
Averaging a burst of fast readings into one stored value reduces noise and file size together.
Checking before you trust it
Plot everything and look for flat lines (a stuck sensor), steps (a restart or a sensor moved), gaps (a power or network failure) and impossible values. Compare a few points with a trusted reference. Keep the raw file untouched and do cleaning in a copy, with a note of what you changed. A graph that looks smooth can hide a lot of problems that the raw data shows.
- Logging only the converted value and losing the ability to correct a calibration later.
- Overwriting the log file at every restart.
- Averaging a biased sensor and believing it is now accurate.
- storage = rate × record size × time
- 10 readings a second at 16 bytes is about 13.8 MB a day
- samples per day = 86 400 ÷ interval
- sets how soon a card or flash fills up
Practise this lesson with Maker
The explanation above is free for everyone. The practice for Intermediate lessons in every domain are part of the Maker plan at ₹499 a month, and Pro includes them too.
With Maker you also get: a worked example, a hands-on task, a 3-question check.
