Learn / NVIDIA Jetson / Cameras and detection

Camera Pipelines on Jetson

ReviewedIntermediate MakerAbout 6 min
NANO
In short

Camera Pipelines on Jetson: the short answer

The Orin Nano Developer Kit has two 22-pin MIPI CSI camera connectors and works with IMX219 cameras such as the Raspberry Pi Camera Module 2 type boards sold for Jetson. py, choose the option to configure the CSI connector, pick your IMX219 camera, save and reboot.

Level
Intermediate
Time
About 6 min
Topic
NVIDIA Jetson
You will learn to

Get frames from a CSI or USB camera into Python and OpenCV efficiently with GStreamer.

Before this: First Boot on a Jetson Orin Nano

The Orin Nano Developer Kit has two 22-pin MIPI CSI camera connectors and works with IMX219 cameras such as the Raspberry Pi Camera Module 2 type boards sold for Jetson. On JetPack 6 you first tell the board which camera is attached: run sudo /opt/nvidia/jetson-io/jetson-io.py, choose the option to configure the CSI connector, pick your IMX219 camera, save and reboot. A USB webcam needs no setup and shows up as /dev/video0.

CSI cameras on Jetson go through NVIDIA's image processor, which you reach with the GStreamer element nvarguscamerasrc. Test with: gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1' ! nvvidconv ! xvimagesink. The memory:NVMM part keeps frames in GPU-friendly memory, and nvvidconv converts and scales them on hardware instead of the CPU.

To use frames in OpenCV, build the same pipeline as a string ending in appsink and open it with cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER). A working string is: nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1. Let the hardware scale the image down to the size your model needs. Note that the Orin Nano has no hardware video encoder, so recording H.264 uses the CPU; record at lower resolution or save only the clips you need.

Move frames without copying them

A camera pipeline is a chain of stages: capture, convert, resize, process and display or store. On a Jetson, GStreamer can pass frames between hardware-accelerated stages without bringing them into the CPU's memory each time. Each extra copy of a large image costs time, so keep the data on the hardware path for as long as you can.

A CSI camera and a USB camera take different routes. A USB camera often delivers compressed frames that must be decoded, which uses more processing than a CSI camera.

Measure frame rate and delay

The frame rate you set is what the camera offers, not what your program achieves. Print the real frames per second and the delay from capture to result. If your processing is slower than the camera, frames queue up and the delay grows, so drop old frames and process the newest.

Practical checks

Test the pipeline with a simple capture and display before adding a model, so you know which stage is slow. Keep resolution as small as the task allows, since a detector often works well at a few hundred pixels across, and watch temperature during long runs.

Common mistakes
  • Copying frames to the CPU at every stage of the pipeline.
  • Believing the requested frame rate without measuring the real one.
  • Letting frames queue until the delay grows large.
Intermediate lesson · Maker plan

Practise this lesson with Maker

The explanation above is free for everyone. The practice for Intermediate lessons in every domain are part of the Maker plan at ₹499 a month, and Pro includes them too.

With Maker you also get: a worked example, a hands-on task, a 3-question check.

Common questions

What will I learn in "Camera Pipelines on Jetson"?

Get frames from a CSI or USB camera into Python and OpenCV efficiently with GStreamer.