{"slug": "setting-up-a-dgx-spark-for-ai-development", "title": "Setting up a DGX Spark for AI Development", "summary": "An engineer detailed the setup of an NVIDIA DGX Spark for AI development, covering system updates, dependency installation, and configuration of Ollama with models like Qwen3.8 and Qwen2.5-Coder. The guide includes steps for enabling LAN access, setting context length, and security considerations for local AI tooling.", "body_md": "[Initial DGX Spark Setup](#initial-dgx-spark-setup)[DGX Spark Dependencies](#dgx-spark-dependencies)[Ollama](#ollama)[OpenCode CLI](#opencode-cli)[AI Coding in IDE](#ai-coding-in-ide)[Open WebUI](#open-webui)[Security Considerations](#security-considerations)[Privacy Considerations](#privacy-considerations)[Optional Hardware Recommendations](#optional-hardware-recommendations)\n\nAfter completing any system updates, the first real thing you would want to do is open the DGX Dashboard application.\n\n- Connect to the DGX Spark using a keyboard, mouse and monitor\n- Click\n`Show Apps`\n\nicon in bottom left - Search for and Open the\n**DGX Dashboard** application - Visit\n**Updates** and apply any updates - Visit\n**Settings** and update a**Hostname**`*`\n\nif desired ( our documentation uses`dgx-spark`\n\nas the Hostname ) and disable**Telemetry** if desired ( recommended )\n\n`*`\n\nYou will need to restart the DGX Spark after changing the Hostname.\n\nYou will need to run the following while connected to the DGX Spark.\n\n```\nsudo apt update\nsudo apt install -y build-essential curl git libbz2-dev libffi-dev liblzma-dev libncursesw5-dev libreadline-dev libsqlite3-dev libssl-dev libxml2-dev libxmlsec1-dev tk-dev uuid-dev xz-utils zlib1g-dev\n```\n\nYou are likely going to want different versions of Python, other than the 3.12 version that comes with the DGX Spark.\n\n```\ncurl https://pyenv.run | bash\n```\n\nThen add pyenv to your shell startup configuration. Since DGX OS normally uses Bash:\n\n```\ncat >> ~/.bashrc <<'EOF'\n\n# pyenv\nexport PYENV_ROOT=\"$HOME/.pyenv\"\n[[ -d $PYENV_ROOT/bin ]] && export PATH=\"$PYENV_ROOT/bin:$PATH\"\neval \"$(pyenv init - bash)\"\nEOF\n```\n\nAlso add it to your login profile:\n\n```\ncat >> ~/.profile <<'EOF'\n\n# pyenv\nexport PYENV_ROOT=\"$HOME/.pyenv\"\n[[ -d $PYENV_ROOT/bin ]] && export PATH=\"$PYENV_ROOT/bin:$PATH\"\nEOF\n```\n\nReload the shell:\n\n```\nexec \"$SHELL\"\n```\n\nThen verify:\n\n```\npyenv --version\n```\n\nConnect to DGX Spark\n\nReplace\n\n`aidev`\n\nwith you actual username and`dgx-spark`\n\nwith your Hostname\n\n```\nssh aidev@dgx-spark.local\n```\n\nInstall/Update Ollama:\n\n```\ncurl -fsSL https://ollama.com/install.sh | sh\n```\n\nVerify Ollama installation:\n\n```\nollama --version\nollama pull qwen3.8:27b\nollama pull qwen2.5-coder:7b\nollama pull nomic-embed-text\n```\n\nVerify Ollama models:\n\n```\nollama list\n```\n\n| Model | Why this model |\n|---|---|\n`qwen3.8:27b` |\nPrimary reasoning/agent model. The ~27B size provides substantially better reasoning, instruction following, code understanding, and multi-step problem solving than a small autocomplete model, while remaining practical to run locally on the DGX Spark. It's intended for chat, code analysis, edits, and agentic work. |\n`qwen2.5-coder:7b` |\nDedicated autocomplete model. Autocomplete needs very low latency and is invoked constantly. A specialized 7B coding model is fast enough for interactive completion while still being strong at predicting code, avoiding the latency and compute cost of invoking the 27B model on every keystroke. |\n`nomic-embed-text` |\nDedicated embedding/retrieval model. It's purpose-built to turn text and source code into embeddings for semantic search. It's small, fast, and inexpensive to keep available, making it a better choice for codebase indexing/retrieval than using a generative LLM. |\n\n```\nsudo systemctl edit ollama\n```\n\nAdd the following:\n\n```\n[Service]\nEnvironment=\"OLLAMA_HOST=0.0.0.0:11434\"\nEnvironment=\"OLLAMA_CONTEXT_LENGTH=262144\"\nEnvironment=\"OLLAMA_KEEP_ALIVE=-1\"\nEnvironment=\"OLLAMA_NUM_PARALLEL=1\"\nEnvironment=\"OLLAMA_MAX_LOADED_MODELS=3\"\nEnvironment=\"OLLAMA_FLASH_ATTENTION=1\"\nEnvironment=\"OLLAMA_KV_CACHE_TYPE=f16\"\nEnvironment=\"OLLAMA_NO_CLOUD=1\"\n```\n\nReload Ollama:\n\n```\nsudo systemctl daemon-reload\nsudo systemctl restart ollama\n```\n\n| Setting | What it does / Why it matters |\n|---|---|\n`OLLAMA_HOST=0.0.0.0:11434` |\nBinds Ollama to all network interfaces instead of localhost only. This allows trusted LAN clients such as the MacBook running VS Code/Continue to reach the DGX Spark. |\n`OLLAMA_CONTEXT_LENGTH=262144` |\nSets the model context window to 262K tokens, matching the full context capacity of Qwen 3.8 27B. This gives coding agents maximum working space for repository context, conversation history, tool definitions, command output, diffs, and retrieved code, reducing the need for context compaction during complex or long-running development tasks. |\n`OLLAMA_KEEP_ALIVE=-1` |\nKeeps loaded models resident in memory indefinitely rather than unloading them after inactivity. This eliminates model reload/cold-start delays during intermittent development work. |\n`OLLAMA_NUM_PARALLEL=1` |\nLimits each model to one concurrent request. For a primarily single-user coding server, this prioritizes memory efficiency and large-context capacity over request concurrency. |\n`OLLAMA_MAX_LOADED_MODELS=3` |\nAllows up to three models to remain loaded simultaneously. This matches the intended workload: primary coding/chat model, autocomplete model, and embedding model. |\n`OLLAMA_FLASH_ATTENTION=1` |\nEnables Flash Attention, an optimized attention implementation that reduces memory usage and can improve performance, particularly with large context windows such as 64K. |\n`OLLAMA_KV_CACHE_TYPE=f16` |\nStores the attention KV cache in 16-bit floating-point form. This uses roughly twice the memory of q8_0, but avoids KV-cache quantization and preserves maximum precision, making it preferable when memory capacity is not a constraint. |\n`OLLAMA_NO_CLOUD=1` |\nDisables Ollama's cloud functionality. This ensures the DGX is configured as a local-only inference server, which is desirable for privacy, security, and predictable data flow. |\n\nNow we are going to preload our largest model into memory on boot. The `qwen3.8:27b`\n\nmodel has the expensive cold-start penalty. The 7B autocomplete model and Nomic embed model are much smaller and will load comparatively quickly, and will remain in memory after being loaded.\n\nFirst, let's create our loader script:\n\n```\nsudo nano /usr/local/bin/ollama-preload.sh\n```\n\nThen enter the following code:\n\n``` bash\n#!/bin/bash\n\nset -e\n\nOLLAMA_URL=\"http://127.0.0.1:11434\"\nMAX_ATTEMPTS=30\nSLEEP_SECONDS=2\n\nfor ((attempt=1; attempt<=MAX_ATTEMPTS; attempt++)); do\n    if curl -sf \"${OLLAMA_URL}/api/tags\" >/dev/null; then\n        echo \"Ollama is ready.\"\n        break\n    fi\n\n    if (( attempt == MAX_ATTEMPTS )); then\n        echo \"ERROR: Ollama did not become ready within 60 seconds.\" >&2\n        exit 1\n    fi\n\n    sleep \"${SLEEP_SECONDS}\"\ndone\n\necho \"Preloading qwen3.8:27b...\"\n\ncurl --fail --silent --show-error \\\n    \"${OLLAMA_URL}/api/generate\" \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\"model\":\"qwen3.8:27b\",\"keep_alive\":-1}'\n\necho \"Model preload complete.\"\n```\n\nNow let's make that executable:\n\n```\nsudo chmod +x /usr/local/bin/ollama-preload.sh\n```\n\nCreate systemd service:\n\n```\nsudo nano /etc/systemd/system/ollama-preload.service\n```\n\nUse the following code:\n\n```\n[Unit]\nDescription=Preload Ollama      \nRequires=ollama.service\nAfter=ollama.service\n\n[Service]\nType=oneshot\nExecStart=/usr/local/bin/ollama-preload.sh\nTimeoutStartSec=120\nRemainAfterExit=yes\n\n[Install]\nWantedBy=multi-user.target\n```\n\nReload System:\n\n```\nsudo systemctl daemon-reload\nsudo systemctl enable ollama-preload.service\nsudo systemctl start ollama-preload.service\n```\n\nOne you have setup your DGX Spark, you can use [OpenCode](https://opencode.ai/) in your terminal on any machine that has access to the DGX Spark.\n\n```\ncurl -fsSL https://opencode.ai/install | bash\n```\n\nThen you can confirm it is installed:\n\n```\nopencode --version\n```\n\nCreate a directory:\n\n```\nmkdir -p ~/.config/opencode\n```\n\nCreate your config file:\n\n```\nnano ~/.config/opencode/opencode.json\n```\n\nPaste in the JSON from [openconfig.json](#file-openconfig-json)\n\nCreate agents folder:\n\n```\nmkdir -p ~/.config/opencode/agents/\n```\n\nCreate agent files inside that folder:\n\nIn your terminal, you can now run:\n\n```\nopencode models ollama\n```\n\nYou should see:\n\n```\nollama/qwen2.5-coder:7b\nollama/qwen3.8:27b\n```\n\nNow you can change to any project directory where you want to work, and run:\n\n```\ncd /path/to/your/project\nopencode\n```\n\nThen to test OpenCode you can just ask a starter question like:\n\n```\nWhat can you tell me about this project?\n```\n\nIf you would like to use your DGX Spark in your IDE:\n\nInstall the [Continue.dev](https://docs.continue.dev/ide-extensions/install) IDE Extension.\n\nCreate your config file:\n\n```\nnano ~/.continue/config.yaml\n```\n\nPaste in the following:\n\n```\nname: DGX Spark\nversion: 1.0.0\nschema: v1\nmodels:\n  - name: Qwen 3.8 27B\n    provider: ollama\n    model: qwen3.8:27b\n    apiBase: http://dgx-spark.local:11434\n    roles:\n      - chat\n      - edit\n      - apply\n    capabilities:\n      - tool_use\n      - image_input\n  - name: Qwen 2.5 Coder 7B\n    provider: ollama\n    model: qwen2.5-coder:7b\n    apiBase: http://dgx-spark.local:11434\n    roles:\n      - autocomplete\n  - name: Nomic Embed\n    provider: ollama\n    model: nomic-embed-text\n    apiBase: http://dgx-spark.local:11434\n    roles:\n      - embed\ncontext:\n  - provider: code\n  - provider: codebase\n  - provider: currentFile\n  - provider: diff\n  - provider: docs\n  - provider: folder\n  - provider: open\n  - provider: problems\n  - provider: terminal\n  - provider: tree\n```\n\nNow all you need to do is Open the Continue Extension in your IDE ( restart your IDE if it was already open ).\n\nConnect to DGX Spark\n\n```\nssh aidev@dgx-spark.local\n```\n\nInstall required version of Python\n\n```\npyenv install 3.11.13\n```\n\nCreate a dedicated Open WebUI environment\n\n```\nmkdir -p ~/apps/open-webui\ncd ~/apps/open-webui\n```\n\nTell pyenv that this directory should use Python 3.11:\n\n```\npyenv local 3.11.13\n```\n\nNow create the virtual environment:\n\n```\npython -m venv .venv\n```\n\nActivate it:\n\n```\nsource .venv/bin/activate\n```\n\nUpgrade packaging tools:\n\n```\npython -m pip install --upgrade pip setuptools wheel\npip install open-webui\n```\n\nTest that this runs as expected:\n\n```\nexport OLLAMA_BASE_URL=\"http://localhost:11434\"\nexport DO_NOT_TRACK=\"true\"\nexport SCARF_NO_ANALYTICS=\"true\"\nexport ANONYMIZED_TELEMETRY=\"false\"\n\nopen-webui serve --host 0.0.0.0 --port 8080\n```\n\nThis DGX Spark configuration is designed for flexible development use across both trusted home networks and direct-wired travel environments. The appropriate security posture depends on how the Spark is connected, with additional firewall protections recommended whenever it must operate on an untrusted or unknown network.\n\nWhen the DGX Spark is connected to a trusted private home Wi-Fi network, a host-level firewall such as UFW is generally optional. The home router/firewall provides the primary security boundary against unsolicited Internet traffic.\n\nFor travel, a direct Ethernet connection between the development laptop and DGX Spark provides a simple private network. This avoids placing the DGX Spark directly on hotel, conference, airport, or other untrusted Wi-Fi networks.\n\nIf the DGX Spark must connect directly to a network that is public, shared, or not fully trusted, enable a host firewall.\n\nWhen administering the Spark remotely, verify that SSH is permitted **BEFORE** enabling UFW to avoid locking yourself out.\n\nUbuntu's UFW provides a straightforward baseline:\n\n```\nsudo ufw default deny incoming\nsudo ufw default allow outgoing\n```\n\nThen explicitly permit only required services. For example:\n\n```\n# SSH\nsudo ufw allow 22/tcp\n\n# Ollama\nsudo ufw allow 11434/tcp\n\n# Example development server\nsudo ufw allow 3000/tcp\n```\n\nEnable and verify the firewall:\n\n```\nsudo ufw enable\nsudo ufw status verbose\n```\n\n**Important**: When default deny incoming is enabled, every remotely accessible service must be explicitly permitted. Developers running dynamic Node.js applications, Vite servers, Jupyter, debugging tools, or other services will need to add rules for their respective ports.\n\nPrevent submitting hardware configuration, RAM/disk sizes, timezone, language, etc.\n\n```\nubuntu-report -f send no\n```\n\nDisable VS Code telemetry\n\n- Open VS Code.\n- Press\n`Cmd`+`,` to open Settings. - In the search box, enter:\n\n```\ntelemetry\n```\n\n- Find Telemetry: Telemetry Level.\n- Set it to:\n\n```\noff\n```\n\nYou an also uncheck everything else you want to disable.\n\n```\n{\n    \"allowAnonymousTelemetry\": false\n}\n```\n\nIn your `~/.config/opencode/opencode.json`\n\nfile:\n\n```\n{\n  \"share\": \"disabled\"\n}\n```\n\n**External USB-C SSD (2–4 TB+)**- Useful for storing training datasets, RAG collections, media, model exports, checkpoints, and other bulk data while reserving the DGX Spark's internal NVMe for frequently accessed models, caches, applications, and performance-sensitive AI workloads.**USB-C to Ethernet Adapter**- Provides the laptop with a dedicated wired Ethernet connection to the DGX Spark when Wi-Fi is unavailable, untrusted, or undesirable. Particularly useful for laptops without built-in Ethernet.**CAT6 Cable (3 ft)**- Enables a simple, fast direct connection between the development laptop and DGX Spark without relying on hotel, conference, or other external network infrastructure.**HDMI Dummy Plug**- A DGX Spark can operate completely headless and does not require a monitor for SSH, Ollama, Jupyter, NVIDIA Sync, or other network-based development workflows. However, an inexpensive HDMI dummy plug can be useful if you plan to use the Ubuntu graphical desktop remotely, as it causes the system to detect a persistent display and can avoid resolution, remote-desktop, or display-session issues sometimes encountered on fully headless systems.**Compact Keyboard w/ Trackpad**- Primarily a recovery tool. Useful if you need local console access and can't restore networking remotely.** USB-C to USB-A Adapter**- The DGX Spark provides USB-C ports but no traditional USB-A ports. A compact adapter is useful for connecting common keyboards, mice, USB flash drives, recovery media, and other legacy USB peripherals, particularly when troubleshooting or performing system recovery.**Compact HDMI Cable**- Worth carrying as a recovery option. If networking, SSH, or remote desktop configuration fails, you can connect the Spark to a hotel TV or other available HDMI display for troubleshooting.", "url": "https://wpnews.pro/news/setting-up-a-dgx-spark-for-ai-development", "canonical_source": "https://gist.github.com/manifestinteractive/48c2fa52cd74a97f115ee9bbddae9df6", "published_at": "2026-08-19 22:58:27+00:00", "updated_at": "2026-08-26 06:13:43.061225+00:00", "lang": "en", "topics": ["artificial-intelligence", "developer-tools", "ai-infrastructure", "large-language-models", "ai-tools"], "entities": ["NVIDIA", "DGX Spark", "Ollama", "Qwen", "pyenv"], "alternates": {"html": "https://wpnews.pro/news/setting-up-a-dgx-spark-for-ai-development", "markdown": "https://wpnews.pro/news/setting-up-a-dgx-spark-for-ai-development.md", "text": "https://wpnews.pro/news/setting-up-a-dgx-spark-for-ai-development.txt", "jsonld": "https://wpnews.pro/news/setting-up-a-dgx-spark-for-ai-development.jsonld"}}