Idea
In short
How do you build a local language model assistant?
Run a small language model on a Jetson Orin so a voice assistant answers without the cloud. You need NVIDIA Jetson Orin Nano Super Developer Kit, NVMe SSD 256 GB (M.2 2230/2242), MAX4466 microphone module and Speaker 8 Ω 0.5 W.
- Level
- Advanced
- Time
- About 480 min
- Parts
- 4
An idea, not a guided build yet. Goal, parts and steps; no wiring, code or tests. Copy it to My Garage to plan it.
Goal
Deploy a language model on edge hardware.
Parts list
- Single Board Computers
- SBC Accessories
- Sensors
- Motors & Actuators
How to build it
- Pick a model that fits in memory
- Quantise it
- Run it with a local server
- Connect speech in and out
- Measure tokens per second
Common questions
What parts do I need for the local language model assistant?
NVIDIA Jetson Orin Nano Super Developer Kit, NVMe SSD 256 GB (M.2 2230/2242), MAX4466 microphone module and Speaker 8 Ω 0.5 W.
How long does the local language model assistant take to build?
About 480 minutes for a advanced maker.
