Learn / NVIDIA Jetson / Setup and first boot

What a Jetson Is and When to Use One

ReviewedBeginnerFreeAbout 6 min
NANO
In short

What a Jetson Is and When to Use One: the short answer

An NVIDIA Jetson is a small Linux computer with an NVIDIA GPU on the same chip. The GPU runs thousands of simple calculations in parallel, which is exactly what neural networks need. Example: A Pi with the AI Camera is enough for one door. Choose the Jetson when you need several cameras, larger models or custom models trained in PyTorch.

Level
Beginner
Time
About 6 min
Topic
NVIDIA Jetson
You will learn to

Understand what a Jetson Orin Nano does that a Raspberry Pi or ESP32 cannot, and decide when it is worth the cost.

Before this: Connected Embedded Devices

An NVIDIA Jetson is a small Linux computer with an NVIDIA GPU on the same chip. The GPU runs thousands of simple calculations in parallel, which is exactly what neural networks need. The Jetson Orin Nano Super Developer Kit has a 6-core Arm CPU, an Ampere GPU with 1,024 CUDA cores and 32 Tensor Cores, and 8 GB of memory shared between CPU and GPU. NVIDIA rates it at up to 67 TOPS for AI, which is enough to run object detection on several camera streams in real time.

The Jetson runs Ubuntu with NVIDIA's JetPack software: CUDA for GPU programming, cuDNN and TensorRT for fast inference, and camera and video libraries. The same tools are used on large NVIDIA servers, so models trained on a PC usually move to a Jetson with little change.

Use a Jetson when the project needs real AI on video: detecting objects, reading number plates, tracking people, or running a small language model locally. Use a Raspberry Pi for general Linux projects, dashboards and light vision. Use an ESP32 or Arduino for sensors, motors and anything battery powered. The Jetson draws 7 to 25 W, costs several times more than a Pi, and its 40-pin header is 3.3 V like the Pi's, so it is usually paired with a microcontroller for hard real-time control.

Why a GPU helps with AI

A neural network is mostly a very large number of multiplications and additions that do not depend on each other. A CPU does a few at a time, very quickly. A GPU does thousands at the same time, more slowly each, and wins easily on this kind of work. That is why a small board with a GPU can run an object detector on live video while a Pi manages only a few frames a second.

Memory matters too. On a Jetson the CPU and GPU share the same memory, so a large model competes with the operating system and your program for it. Check a model's memory use before you plan around it.

Power, heat and cost are part of the decision

A Jetson works at several watts up to a couple of tens of watts depending on its power mode, so it needs a proper power supply, a fan or good airflow, and an enclosure that does not trap heat. It is not a battery-friendly part like an ESP32. The kit also costs many times a Raspberry Pi.

Before you buy, measure what you need. Often a Pi with a small, efficient model, or a microcontroller with a tiny one, is enough. A Jetson is worth the price when you need several cameras, larger models or real-time speed that the cheaper boards cannot reach.

The software stack

You write in Python or C++ and use the libraries that come with NVIDIA's software package for the board: CUDA for general GPU programming, and tools that convert a trained model into a form that runs fast on the GPU. Containers are common, because they package a model's many dependencies together and run the same way on every board.

Training a model is usually done on a PC or in the cloud; the Jetson's job is to run it. That keeps the board small and the project simple.

Pairing it with a microcontroller

The Jetson is good at seeing and deciding; it is not good at tight timing or at running from a small battery. A common pattern is a camera on the Jetson, a model that finds the object, and a message over USB serial to an Arduino or ESP32 that moves the motor or switches the relay. Each board does what it is best at.

Common mistakes
  • Buying a Jetson before measuring whether a cheaper board can run the model.
  • Running it in a closed box with no airflow.
  • Driving motors or high-current loads straight from its 3.3 V header pins.
Worked example

Pick the board for a shop-door customer counter

  1. One camera, people only, results needed a few times a second.
  2. A Raspberry Pi AI Camera runs person detection on-sensor at good frame rates.
  3. A Jetson Orin Nano could also do it, plus several more cameras or a bigger model.
  4. Cost and power: the Pi option is cheaper and uses about 5 W; the Jetson uses up to about 25 W.

Result. A Pi with the AI Camera is enough for one door. Choose the Jetson when you need several cameras, larger models or custom models trained in PyTorch.

Try it

Write down one AI project idea, then fill in three columns: frames per second needed, number of cameras, and power source. Decide whether it needs a Jetson, a Pi with an AI camera, or a microcontroller, and note why.

Check yourself

3 quick questions

  1. What makes a Jetson fast at AI?

  2. Which software stack comes with the Jetson?

  3. A battery-powered soil sensor that reports once an hour needs which board?

Common questions

What will I learn in "What a Jetson Is and When to Use One"?

Understand what a Jetson Orin Nano does that a Raspberry Pi or ESP32 cannot, and decide when it is worth the cost.

How do I pick the board for a shop-door customer counter?

One camera, people only, results needed a few times a second. A Raspberry Pi AI Camera runs person detection on-sensor at good frame rates. A Jetson Orin Nano could also do it, plus several more cameras or a bigger model. Cost and power: the Pi option is cheaper and uses about 5 W; the Jetson uses up to about 25 W. A Pi with the AI Camera is enough for one door. Choose the Jetson when you need several cameras, larger models or custom models trained in PyTorch.

What makes a Jetson fast at AI?

An NVIDIA GPU with Tensor Cores on the same chip. Neural networks run on the GPU and Tensor Cores in parallel.

Which software stack comes with the Jetson?

JetPack: Ubuntu, CUDA, cuDNN and TensorRT. JetPack bundles the OS and NVIDIA's AI libraries.

A battery-powered soil sensor that reports once an hour needs which board?

ESP32. Low-power sensing is microcontroller work; a Jetson would drain the battery quickly.