Training a Custom Detector for Jetson: the short answer
Pretrained models detect common objects such as people and cars. For anything specific (ripe tomatoes, a missing screw, a helmet on a worker) you train your own detector.
- Level
- Intermediate
- Time
- About 6 min
- Topic
- NVIDIA Jetson
Collect and label images, train a custom YOLO or SSD detector, and run it on the Jetson with TensorRT.
Before this: Object Detection with Pretrained Models on Jetson
Pretrained models detect common objects such as people and cars. For anything specific (ripe tomatoes, a missing screw, a helmet on a worker) you train your own detector. Start with the data: collect a few hundred images per class from the real camera, in the real place, across lighting, angles and backgrounds you will see. Include images with no objects too. Label each object with a box using a tool such as Label Studio, CVAT or Roboflow, and export in the format your training tool expects, such as YOLO text files or Pascal VOC.
Train on a PC with an NVIDIA GPU or in a cloud notebook. With the Ultralytics YOLO tools, transfer learning from a small pretrained model (the n or s sizes) needs only a few commands, and a few hundred epochs on a few hundred images often take under an hour on a cloud GPU. Keep 15 to 20 % of images for validation, and watch the mAP score and the validation loss; stop when they stop improving. NVIDIA's jetson-inference project can also retrain an SSD-MobileNet detector on the Jetson itself, which is slower but needs nothing else.
To deploy, export the trained model to ONNX, then build a TensorRT engine on the Jetson itself, because engines are tied to the exact GPU and TensorRT version; with Ultralytics, run the export to the engine format on the Jetson, and use FP16 for a good balance of speed and accuracy. Test the engine on fresh images from the camera, measure the frames per second, and set the confidence threshold from real results. When it misses objects in some conditions, add those images to the dataset and retrain; that loop is how real detectors get good.
Collect and label the right images
A custom detector learns from images you label with boxes around each object. Take pictures with the real camera in the real places, in different lighting, distances and backgrounds, because the model learns whatever is consistent in the data, including accidental things such as a background that is always the same.
A few hundred well-chosen images per class is a reasonable start for a simple object. Add images with no objects as well, so the model learns what to ignore.
Train, then test on new images
Fine-tuning a pretrained model needs far less data than starting from nothing. Keep a validation set that the model never trains on and check its results. If the model does well on training images but badly on validation ones, it has memorised the training set, and you need more varied images.
Look at the failures. Missed detections and false ones show which situations are missing from your data, and the fix is usually more pictures of exactly those cases.
Deploy and keep improving
Export the model for the Jetson, optimise it, and test it live. Save frames the model gets wrong, label them, and add them to the next training round. A detector improves most from a loop of real mistakes added to the data.
- Training on images that all share the same background.
- Judging the result by training images instead of unseen ones.
- Never adding the model's real-world mistakes back into the data.
Practise this lesson with Maker
The explanation above is free for everyone. The practice for Intermediate lessons in every domain are part of the Maker plan at ₹499 a month, and Pro includes them too.
With Maker you also get: a worked example, a hands-on task, a 3-question check.
