Learn / NVIDIA Jetson / Optimise and deploy

Multi-Camera Video Analytics with DeepStream

ReviewedAdvanced ProAbout 6 min
NANO
In short

Multi-Camera Video Analytics with DeepStream: the short answer

DeepStream is NVIDIA's toolkit for video analytics, built on GStreamer. A pipeline is a chain of plugins: sources (RTSP IP cameras, files or CSI cameras), hardware decoding (nvv4l2decoder), nvstreammux, which batches frames from several streams together, nvinfer, which runs a TensorRT model on the whole batch, nvtracker, which follows each object across frames and gives it an ID, and nvdsosd, which draws boxes.

Level
Advanced
Time
About 6 min
Topic
NVIDIA Jetson
You will learn to

Build a DeepStream pipeline that decodes several camera streams, batches them through one model, tracks objects and sends results out.

Before this: Optimising Models with TensorRT, Camera Pipelines on Jetson

DeepStream is NVIDIA's toolkit for video analytics, built on GStreamer. A pipeline is a chain of plugins: sources (RTSP IP cameras, files or CSI cameras), hardware decoding (nvv4l2decoder), nvstreammux, which batches frames from several streams together, nvinfer, which runs a TensorRT model on the whole batch, nvtracker, which follows each object across frames and gives it an ID, and nvdsosd, which draws boxes. Message plugins can send the results as JSON over MQTT or Kafka instead of, or as well as, showing video.

Batching is the key to performance. Running one model on a batch of four frames is much faster than four separate inferences, so set the muxer's and nvinfer's batch-size to the number of streams. Set the muxer's width and height to the model's input size so frames are scaled once. Decoding on the dedicated hardware decoder keeps the CPU free. Note that the Orin Nano has no hardware video encoder, so re-encoding output video uses the CPU; for multi-camera systems, send metadata rather than annotated video, or record only short clips around events.

Start from the reference application (deepstream-app) and its text configuration files, which set up sources, the muxer, the model and the tracker without writing code; the Python bindings let you attach a probe to read each frame's objects and apply your own rules. Measure throughput per stream and watch GPU and memory use with jtop. Plan capacity honestly: a model that runs at 120 fps on one stream runs about 30 fps per stream on four, and you may need to skip frames (the nvinfer interval setting) and let the tracker fill the gaps. Store as little video as possible and follow privacy law and consent requirements when cameras see people.

A pipeline built on GStreamer

DeepStream is a set of GStreamer plug-ins for analysing video. A typical pipeline reads several sources, joins them into one batch with the stream multiplexer, runs the detector on the batch, tracks objects between frames, and outputs the results as on-screen boxes, a file or messages. Batching several streams through the model together uses the GPU far better than running them one at a time.

A configuration file sets the model, the batch size, the confidence thresholds and the tracker, so you can change them without rewriting code.

Scaling and measuring

The total load is the number of streams times the frame rate times the resolution. Measure the frames per second per stream under the full load, and watch for the point where it drops below the source rate. Reduce the resolution, skip frames for the detector and run the tracker between detections, or use a smaller model to fit more streams.

Decoding the video itself uses hardware, so check that the cameras' codec and resolution are supported.

Outputs

Send results onward as small messages, for example to MQTT, instead of sending video. Handle a stream that drops by reconnecting without stopping the others. For privacy, store only the events or counts you need.

Common mistakes
  • Running each camera through its own copy of the model instead of batching.
  • Not measuring the real per-stream frame rate under full load.
  • Letting one failed camera stop the whole pipeline.
Advanced lesson · Pro plan

Practise this lesson with Pro

The explanation above is free for everyone. The practice for Advanced lessons in every domain are part of the Pro plan at ₹999 a month.

With Pro you also get: a worked example, a hands-on task, a 3-question check.

Common questions

What will I learn in "Multi-Camera Video Analytics with DeepStream"?

Build a DeepStream pipeline that decodes several camera streams, batches them through one model, tracks objects and sends results out.