Multi-Camera Video Analytics with DeepStream: the short answer
DeepStream is NVIDIA's toolkit for video analytics, built on GStreamer. A pipeline is a chain of plugins: sources (RTSP IP cameras, files or CSI cameras), hardware decoding (nvv4l2decoder), nvstreammux, which batches frames from several streams together, nvinfer, which runs a TensorRT model on the whole batch, nvtracker, which follows each object across frames and gives it an ID, and nvdsosd, which draws boxes.
- Level
- Advanced
- Time
- About 6 min
- Topic
- NVIDIA Jetson
Build a DeepStream pipeline that decodes several camera streams, batches them through one model, tracks objects and sends results out.
Before this: Optimising Models with TensorRT, Camera Pipelines on Jetson
DeepStream is NVIDIA's toolkit for video analytics, built on GStreamer. A pipeline is a chain of plugins: sources (RTSP IP cameras, files or CSI cameras), hardware decoding (nvv4l2decoder), nvstreammux, which batches frames from several streams together, nvinfer, which runs a TensorRT model on the whole batch, nvtracker, which follows each object across frames and gives it an ID, and nvdsosd, which draws boxes. Message plugins can send the results as JSON over MQTT or Kafka instead of, or as well as, showing video.
Batching is the key to performance. Running one model on a batch of four frames is much faster than four separate inferences, so set the muxer's and nvinfer's batch-size to the number of streams. Set the muxer's width and height to the model's input size so frames are scaled once. Decoding on the dedicated hardware decoder keeps the CPU free. Note that the Orin Nano has no hardware video encoder, so re-encoding output video uses the CPU; for multi-camera systems, send metadata rather than annotated video, or record only short clips around events.
Start from the reference application (deepstream-app) and its text configuration files, which set up sources, the muxer, the model and the tracker without writing code; the Python bindings let you attach a probe to read each frame's objects and apply your own rules. Measure throughput per stream and watch GPU and memory use with jtop. Plan capacity honestly: a model that runs at 120 fps on one stream runs about 30 fps per stream on four, and you may need to skip frames (the nvinfer interval setting) and let the tracker fill the gaps. Store as little video as possible and follow privacy law and consent requirements when cameras see people.
A pipeline built on GStreamer
DeepStream is a set of GStreamer plug-ins for analysing video. A typical pipeline reads several sources, joins them into one batch with the stream multiplexer, runs the detector on the batch, tracks objects between frames, and outputs the results as on-screen boxes, a file or messages. Batching several streams through the model together uses the GPU far better than running them one at a time.
A configuration file sets the model, the batch size, the confidence thresholds and the tracker, so you can change them without rewriting code.
Scaling and measuring
The total load is the number of streams times the frame rate times the resolution. Measure the frames per second per stream under the full load, and watch for the point where it drops below the source rate. Reduce the resolution, skip frames for the detector and run the tracker between detections, or use a smaller model to fit more streams.
Decoding the video itself uses hardware, so check that the cameras' codec and resolution are supported.
Outputs
Send results onward as small messages, for example to MQTT, instead of sending video. Handle a stream that drops by reconnecting without stopping the others. For privacy, store only the events or counts you need.
- Running each camera through its own copy of the model instead of batching.
- Not measuring the real per-stream frame rate under full load.
- Letting one failed camera stop the whole pipeline.
Practise this lesson with Pro
The explanation above is free for everyone. The practice for Advanced lessons in every domain are part of the Pro plan at ₹999 a month.
With Pro you also get: a worked example, a hands-on task, a 3-question check.
