Orin Desktop AI Setup

Transform your NVIDIA Jetson Orin into a fully functional desktop AI workstation — from initial JetPack flashing to running LLMs and vision models locally.

Why Set Up Orin as a Desktop AI Machine?

Most people think of Jetson as an embedded platform — something you cross-compile for, deploy to, and manage remotely. But the Orin AGX, with its 12-core CPU, 64 GB RAM, and Ampere GPU, is more than capable as a primary desktop AI workstation. Setting up an Orin desktop AI setup gives you a silent, low-power development environment where you can write, test, and run AI code directly on the target hardware — no cross-compilation headaches, no deployment friction.

This approach is particularly valuable for Jetson private AI projects where you want to develop and run everything locally. You get the full development experience — IDE, browser, terminal, Docker — on the same machine that serves your AI models. It's a clean, self-contained workflow that many developers find more productive than the traditional edit-on-workstation, deploy-to-Jetson cycle.

Before diving into the setup, make sure you've reviewed the Jetson Orin AI hardware specifications to confirm your module has sufficient memory and storage for your planned workloads. For desktop AI use, we strongly recommend at least the Orin NX 16GB — the Nano's 4–8 GB is workable but tight for modern development environments.

Prerequisites: Jetson Orin AGX or NX developer kit, NVMe SSD (256 GB+), USB keyboard/mouse, HDMI monitor, Ethernet or WiFi connectivity, and a separate computer for SDK Manager flashing (unless you're comfortable with command-line flashing).

Step 1: Flash JetPack with SDK Manager

The first step in any Orin setup is flashing the JetPack SDK — NVIDIA's software stack that includes the Linux for Tegra (L4T) kernel, CUDA, cuDNN, TensorRT, and multimedia APIs. The recommended approach uses NVIDIA SDK Manager on an Ubuntu host machine connected to the Orin via USB-C.

    # On your Ubuntu host machine, install SDK Manager:

    $ wget https://developer.nvidia.com/downloads/sdk-manager

    $ sudo apt install ./sdkmanager_*.deb

    $ sdkmanager

    # Select: Jetson Orin AGX/NX → JetPack 6.x → Flash

    # Connect Orin via USB-C in recovery mode (hold REC button + power)

SDK Manager handles the entire flashing process — bootloader, kernel, root filesystem, CUDA toolkit, and AI libraries. After flashing, the Orin boots into Ubuntu 22.04 with full desktop environment. The initial setup includes display configuration, keyboard layout, language, and user account creation.

Step 2: Configure NVMe Storage

Flashing JetPack to the internal eMMC leaves your NVMe drive unused. For a desktop AI setup, you want JetPack on NVMe for faster boot times, better application performance, and more storage for models and datasets. NVIDIA provides tools to migrate the root filesystem from eMMC to NVMe.

    # After first boot from eMMC, format and mount NVMe:

    $ sudo mkfs.ext4 /dev/nvme0n1

    $ sudo mount /dev/nvme0n1 /mnt

    # Clone root filesystem to NVMe:

    $ sudo rsync -axHAWX --numeric-ids --info=progress2 / /mnt/

    # Update /mnt/boot/extlinux/extlinux.conf root= to point to NVMe

    # Reboot — Orin will boot from NVMe going forward

After migration, you'll have a fast NVMe-based system with the eMMC available as secondary storage or for recovery purposes. A 1 TB NVMe SSD provides ample space for the OS, development tools, Docker images, and dozens of quantized LLM models.

Pro Tip: Use the eMMC as a dedicated swap partition or Docker storage directory after migrating to NVMe. The 64 GB eMMC is fast enough for swap and container layers, freeing your NVMe for model files and datasets.

Step 3: Install Docker and NVIDIA Container Runtime

Docker with NVIDIA GPU support is essential for modern AI workflows. It lets you run pre-built containers from NGC (NVIDIA GPU Cloud) that include optimized versions of PyTorch, TensorFlow, llama.cpp, Ollama, and dozens of other AI tools — all configured for the aarch64 architecture with CUDA backend.

    # Install Docker:

    $ curl -fsSL https://get.docker.com | sudo sh

    $ sudo usermod -aG docker $USER

    # Install NVIDIA Container Toolkit (JetPack includes nvidia-docker2):

    $ sudo apt install -y nvidia-container-toolkit nvidia-docker2

    $ sudo systemctl restart docker

    # Verify GPU access from Docker:

    $ docker run --rm --runtime=nvidia nvcr.io/nvidia/l4t-ml:r36.4.0-py3 nvidia-smi

Once Docker is configured, you can pull and run AI workloads instantly. For Orin self-hosted AI setups, Docker also provides process isolation and resource management — critical when running multiple AI services on the same device.

Step 4: Install Ollama for Local LLMs

Ollama provides the easiest path to running LLMs on Jetson Orin. It handles model downloading, quantization, and serving with a simple CLI and REST API. The Jetson-optimized Ollama build leverages the GPU via CUDA for token generation.

    # Install Ollama (Jetson aarch64 build):

    $ curl -fsSL https://ollama.com/install.sh | sh

    # Pull and run a 7B model:

    $ ollama pull llama3.2:7b-q4_K_M

    $ ollama run llama3.2:7b-q4_K_M

    # API endpoint available at http://localhost:11434

Ollama on Orin AGX achieves 15–25 tokens/second with 7B parameter models at Q4 quantization — perfectly usable for interactive chat. For smaller models like Phi-4-mini or Gemma-2-2B, token rates exceed 40 tokens/second, making local AI assistants genuinely practical.

Step 5: Desktop Environment and Development Tools

A proper Orin desktop AI setup needs a comfortable development environment. Ubuntu 22.04 ships with GNOME, but you can switch to lighter options like XFCE or i3 if you want to reserve more RAM for models. Essential development tools include VS Code (available for aarch64 via code-server or the official arm64 build), Python with Jupyter, and git.

    # Install development essentials:

    $ sudo apt install -y build-essential python3-pip python3-venv git curl htop

    # Install VS Code (arm64):

    $ wget -O code.deb https://code.visualstudio.com/sha/download?build=stable&os=linux-deb-arm64

    $ sudo dpkg -i code.deb

With VS Code, the Python extension, and the Docker extension, you have a complete AI development workstation. The Jetson Orin guide series covers advanced IDE configurations for remote development workflows if you prefer to keep the Orin headless.

Step 6: GPU Monitoring and Power Management

Understanding your Orin's GPU utilization is critical for optimizing AI workloads. The tegrastats utility provides real-time GPU, CPU, and memory statistics. For persistent monitoring, integrate with Prometheus + Grafana or use nvtop for a terminal-based GPU monitor.

    # Real-time GPU/CPU stats:

    $ sudo tegrastats

    # Install nvtop for GPU process monitor:

    $ sudo apt install -y nvtop

    $ nvtop

Power mode selection is handled through nvpmodel. For desktop use, MAXN mode (mode 0) provides best performance; for Orin persistent server use, 30W mode (mode 2) offers an excellent balance of throughput and power efficiency for 24/7 operation.

Alternative: ClawBox Turnkey Solution

If the manual setup process seems daunting, the ClawBox vs Jetson Orin comparison is worth examining. ClawBox ships with everything pre-configured — JetPack flashed, Ollama installed, models pre-loaded, Docker ready — so you can start using AI immediately without the setup overhead. The trade-off is higher cost and less customization flexibility compared to a DIY ClawBox Orin comparison build. For developers who want to understand every layer of their stack, the manual approach documented here provides that depth.

Next Steps

With your Orin desktop AI setup complete, explore our guides on model selection (Topic 3), production deployment (Topic 4), and advanced configuration (Guides 1–3). The Orin AI server path extends your desktop setup into a network-accessible AI service, while the NVIDIA Orin developer portal provides official reference documentation for deeper dives into specific JetPack components.