{"slug": "running-goose-fully-offline-on-a-dgx-spark", "title": "Running goose fully offline on a DGX Spark", "summary": "The open-source agent framework Goose, built at Block and now hosted by the Agentic AI Foundation under the Linux Foundation, can run entirely offline on an NVIDIA DGX Spark with 128 GB of unified memory, using a local Ollama runtime and a model that supports tool calling, with no cloud API calls. A hands-on test confirmed the setup works with zero network packets leaving the machine, proving that local-first agent workflows are feasible on a desk-sized, low-power device.", "body_md": "*A hands-on look at running a local-first agent on the NVIDIA DGX Spark: zero cloud calls, on a 128 GB box that fits on a desk.*\n\nAlmost every agent demo you see talks to a frontier model over a cloud API. It works, but it quietly assumes your code, your prompts, and your data are leaving the building. [goose](https://github.com/aaif-goose/goose), the open-source agent framework built at Block and now hosted by the [Agentic AI Foundation](https://aaif.io/) under the Linux Foundation, does not have to work that way. It is local-first: point it at a model running on your own hardware and the whole loop, reasoning, tool calls, file edits, shell commands, can run without a single packet leaving the machine.\n\nI wanted to test the strongest version of that claim. Take a real agent, point it at a model running locally, give it actual work to do, and then prove that nothing went to the cloud. The machine for the job is the NVIDIA DGX Spark: a GB10 Grace Blackwell box with 128 GB of unified memory that sits on a desk and draws less power than a space heater.\n\nHere is how it went.\n\n## The setup in one picture\n\n*goose talks only to a local Ollama runtime over loopback on a DGX Spark; no cloud API is involved.*\n\nThree pieces, all on one box:\n\n**Ollama** is the model runtime. NVIDIA and Ollama partnered so it runs well on the Spark out of the box, and it exposes an OpenAI-compatible API on localhost:11434.**goose** is the agent. It plans, calls tools, edits files, and runs commands. Its Developer extension is what turns “chat” into “actually does things.”**The model** is whatever you load with Ollama, as long as it supports tool calling. That last requirement is not optional, and it is the thing people most often get wrong.\n\nThe only network hop in the entire loop is the loopback interface between Goose and Ollama. We prove that at the end.\n\n## First, meet the box\n\nBefore trusting any benchmark, it is worth confirming what we are actually running on. The Spark is an ARM machine, so uname reports aarch64, and `nvidia-smi`\n\nconfirms the GPU: a single NVIDIA GB10 on driver 580, CUDA 13.\n\n`$ nvidia-smi`\n\nGPU Index |\nName |\nPersistence Mode |\nBus ID |\nDisplay Active |\nVolatile Uncorr. ECC |\nFan |\nTemp |\nPerf State |\nPower Usage / Cap |\nMemory Usage |\nGPU Util |\nCompute Mode |\n0 |\nNVIDIA GB10 | On | `0000000F:01:00.0` |\nOff | N/A | N/A | 45°C | P8 | 4W / N/A | Not Supported | 0% | Default |\n\nOne detail surprises people: the Memory-Usage column reads Not Supported. That is not a broken driver. The GB10 is a unified-memory design, so the GPU has no separate framebuffer to report. The CPU and GPU share one coherent pool of LPDDR5X, and you read it with an ordinary Linux tool:\n\n`free -h`\n\nThe spec sheet says 128 GB; free reports what is left after firmware and OS overhead, as a single pool that both the CPU and the GB10 draw from:\n\n``` bash\n$ free -h\ntotal           used        free      shared  buff/cache   available\nMem:            121Gi       3.8Gi       116Gi       6.6Mi       2.8Gi       117Gi\nSwap:           15Gi          0B        15Gi\n```\n\nThat is 121 GB of usable unified memory. With nothing loaded it is almost entirely free, and loading qwen3.5:35b-a3b later takes only about 29 GB of it. This is the number that matters for “what model can I fit,” and it is why a desk-sized box can hold models that would never fit on a normal GPU.\n\n## A model that can actually use tools\n\nOllama is an easy docker-like way to run the open source models. A quick check of the version and the models already on disk:\n\n``` bash\n$ ollama -v\nollama version is 0.30.10\n\n$ ollama list\nNAME                       ID              SIZE      MODIFIED\nllama3.1:8b                46e0c10c039e    4.9 GB    5 weeks ago\nqwen3.5:0.8b-bf16          81a0b6550a2d    1.8 GB    7 weeks ago\ngemma4:26b                 5571076f3d70    17 GB     2 months ago\ngemma4:latest              c6eb396dbd59    9.6 GB    2 months ago\nqwen2.5vl:7b               5ced39dfa4ba    6.0 GB    3 months ago\nnemotron-3-super:latest    95acc78b3ffd    86 GB     3 months ago\nqwen3.5:35b-a3b            3460ffeede54    23 GB     3 months ago\n```\n\nHere is the catch that trips most people up: goose leans hard on tool calling, so the model must support it. A model without tool calling can only chat, which means disabling every extension and giving up the whole point of an agent.\n\nThat requirement narrows the field. From the list above:\n\n`qwen3.5:35b-a3b`\n\nis the one I reached for. The Qwen3 lineage has excellent tool calling, and its mixture-of-experts design (the a3b means only about 3B parameters are active per token) keeps it fast despite being a 35B-class model. At 23 GB it leaves most of the Spark’s memory free.`llama3.1:8b`\n\nis a great lighter alternative. Llama 3.1 has native tool calling, and at under 5 GB the agent turns feel snappy.\n\n### Don’t trust the model card, test it\n\nModel cards are optimistic. The honest check is to ask the model to call a tool through Ollama’s API and see whether it actually returns a `tool_calls`\n\nfield:\n\n```\ncurl -s http://localhost:11434/api/chat -d '{\n\"model\": \"qwen3.5:35b-a3b\",\n\"stream\": false,\n\"messages\": [{\"role\":\"user\",\"content\":\"What is the weather in Paris? Use the tool.\"}],\n\"tools\": [{\"type\":\"function\",\"function\":{\n\"name\":\"get_weather\",\n\"description\":\"Get current weather for a city\",\n\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}},\"required\":[\"city\"]}\n}}]\n}' | jq '.message'\n```\n\nThe model came back with an empty content and a populated tool_calls. It chose to call the tool, with the right function and the right argument:\n\n```\n{\n  \"role\": \"assistant\",\n  \"content\": \"\",\n  \"thinking\": \"The user wants to know the weather in Paris and has specifically asked me to use a tool ... I should call the get_weather function with \\\"Paris\\\" as the city parameter.\",\n  \"tool_calls\": [\n    {\n      \"id\": \"call_hsv7poun\",\n      \"function\": {\n        \"name\": \"get_weather\",\n        \"arguments\": { \"city\": \"Paris\" }\n      }\n    }\n  ]\n}\n```\n\nThat is exactly what an agent-capable model should do. If instead you get only plain text in content and no tool_calls, that model cannot drive goose. Pick another and run the test again. While we are here, a quick look at raw decode speed. The –verbose flag makes Ollama print timing, including the eval rate in tokens per second:\n\n``` bash\n$ ollama run qwen3.5:35b-a3b --verbose \"What is an AI agent? Answer in one sentence.\"\nAn AI agent is an autonomous system powered by artificial intelligence that can\nperceive its environment, make reasoned decisions, and take independent actions\nto achieve specific goals without constant human intervention.\n\ntotal duration:       25.884s\nload duration:         8.914s\nprompt eval count:        20 token(s)\nprompt eval rate:     146.02 tokens/s\neval count:             1283 token(s)\neval duration:        16.830s\neval rate:             76.23 tokens/s\n```\n\nAbout 76 tokens per second of decode on a desk-sized box. ollama ps confirms where it ran and how much memory it took:\n\n``` bash\n$ ollama ps\nNAME              ID              SIZE     PROCESSOR    CONTEXT    UNTIL\nqwen3.5:35b-a3b   3460ffeede54    29 GB    100% GPU     262144     4 minutes from now\n</pre\n\nThe 100% GPU is the part that matters: none of the model spilled to the CPU. The 29 GB footprint (a little above the 23 GB on disk, once the context window is allocated) still leaves the large majority of the Spark’s 128 GB free.\n```\n\n## Installing goose\n\ngoose installs with a single script. Note the source: it now lives under the aaif-goose GitHub organization, the project’s new home since it joined the Agentic AI Foundation. `curl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash`\n\nThe install script detects the architecture (the Spark is aarch64) and drops the binary in ~/.local/bin/. Add that to your PATH and check the version:\n\n``` bash\n$ export PATH=\"$HOME/.local/bin:$PATH\"\n$ goose --version\n1.39.0\n```\n\n## Wiring goose to the local model\n\nOne interactive command points goose at a model:\n\n`goose configure`\n\nThe first screen asks how you want to set up your provider, and it defaults to a cloud option:\n\n```\nHow would you like to set up your provider?\n> OpenRouter Login (Recommended)\nTetrate Agent Router Service Login\nManual Configuration\n```\n\nThe first two are cloud routing services, the opposite of what we are after. For a fully local setup, arrow down to Manual Configuration and press Enter. From there, choose Ollama as the provider, point it at http://localhost:11434, and enter the model name `qwen3.5:35b-a3b`\n\n.\n\n*Configuring goose for the local Ollama provider. The default nudges you toward a cloud router; Manual Configuration keeps everything on the box.*\n\nThat is the entire configuration. No API key, no account, no .env pointing at a cloud endpoint. Goose is now wired to a model that lives entirely on this machine.\n\n## Giving it real work\n\nChat is not the point. The point is an agent that reads, writes, and runs things. So I started a fresh Goose session and gave it a small, deterministic task.\n\n`goose session`\n\n`Create ~/primes.py`\n\nthat prints the first 20 prime numbers, then run it with python3 and show the output.\n\nThis time goose did exactly the right thing: it wrote a seven-line script, called the shell to run it, and reported the correct result (2, 3, 5, 7 … 71). That is the full agent loop, write then execute then report, running entirely on local hardware.\n\n*Scoped down to a deterministic task, the agent loop runs cleanly: write the file, run it through the shell, report the result.*\n\nThe honest takeaway is that a local agent on the Spark is dependable for well-scoped tasks and shaky on long autonomous chains with a model this size. The headline claim of this post, that all of it runs offline, does not depend on that distinction. But it is worth knowing before you point a local agent at something complex.\n\n## Proving it stayed offline\n\nThis is the headline claim, so it deserves a demonstration rather than an assertion. The clean proof is to show that every network socket in the loop is loopback. While a model is responding, in another terminal:\n\n``` bash\n$ ss -tlnp | grep 11434\nLISTEN 0   4096   127.0.0.1:11434   0.0.0.0:*\n\n$ sudo lsof -i -nP | grep -i ollama\nollama     2697   ollama   4u  IPv4   43329   0t0  TCP 127.0.0.1:11434 (LISTEN)\nollama     2697   ollama   7u  IPv4  745810   0t0  TCP 127.0.0.1:11434->127.0.0.1:59280 (ESTABLISHED)\nollama     2697   ollama   8u  IPv4  745811   0t0  TCP 127.0.0.1:11434->127.0.0.1:59292 (ESTABLISHED)\nllama-ser  256570 ollama  16u  IPv4  744848   0t0  TCP 127.0.0.1:44777 (LISTEN)\n```\n\nRead it line by line. Ollama listens on 127.0.0.1:11434, not 0.0.0.0, so it is not even reachable from another machine. The two established connections are 127.0.0.1 -> 127.0.0.1: goose talking to Ollama, both ends on the same box. And llama-ser, the model server Ollama spawns to run the weights, is bound to loopback too. Not one socket in the path has a routable address.\n\nThe only connection that ever leaves this machine is the SSH session I am using to drive it, and that is incidental: sit down at the Spark with a keyboard and even that disappears. The agent and the model never open a socket that leaves 127.0.0.1. Pull the Ethernet cable mid-task and nothing changes, because there was never anything on the other end to begin with.\n\n## The numbers\n\n| Metric | Value |\n|---|---|\n| Model | qwen3.5:35b-a3b (23 GB on disk) |\n| Footprint when loaded (ollama ps) | 29 GB (100% GPU) |\n| Raw decode rate (ollama –verbose) | 76.23 tokens/s |\n| Time for Goose to write and run the primes task | 3m 48s |\n| Peak GPU compute utilization (sm) | ~91% |\n| External network calls during the run | 0 |\n\n## What this proves, and what it does not\n\nIt proves that an agent doing real multi-step work, writing a file, running it through the shell, observing the result, and reporting back, can run end to end on a single desk-sized machine with nothing leaving the box. For anyone in an air-gapped environment, working with sensitive code, or simply not wanting their repository streamed to a third party, this is a usable setup today, not someday.\n\nIt does not prove that a local model of this size matches a frontier cloud model. The prime-number task is intentionally small and deterministic. It shows that the local agent loop works, but it does not prove that the same setup will reliably handle long, ambiguous, autonomous workflows.\n\nThe honest framing is this: the privacy story is excellent, the workflow is genuinely usable for bounded tasks, and the agentic reliability gap on complex work is still real. The Spark’s 128 GB of unified memory also gives you room to try larger or more capable local models when the task needs it, without changing the local-first architecture.\n\nThat combination of an open agent framework, a local-first design, and hardware that finally makes local models practical is what turns “local agent” from a demo into something you would actually run.\n\n## Try it yourself\n\nThe whole sequence, top to bottom:\n\n**1. Confirm the hardware (memory is unified, so read it with free -h)**\n\n`uname -m && nvidia-smi && free -h`\n\n**2. Pick a tool-calling model and verify it actually calls tools**\n\n```\nollama list\ncurl -s http://localhost:11434/api/chat -d '{\"model\":\"qwen3.5:35b-a3b\",\"stream\":false,\n\"messages\":[{\"role\":\"user\",\"content\":\"weather in Paris? use the tool\"}],\n\"tools\":[{\"type\":\"function\",\"function\":{\"name\":\"get_weather\",\n\"parameters\":{\"type\":\"object\",\"properties\":{\"city\":{\"type\":\"string\"}}}}}]}' | jq '.message'\n```\n\n**3. Install the agent**\n\n```\ncurl -fsSL https://github.com/aaif-goose/goose/releases/download/stable/download_cli.sh | bash\nexport PATH=\"$HOME/.local/bin:$PATH\"\ngoose --version\n```\n\n**4. Wire Goose to the local model**\n\n`goose configure # Manual Configuration -> Ollama -> http://localhost:11434 -> qwen3.5:35b-a3b`\n\n**5. Give it real work**\n\n`goose session`\n\n**6. Prove it stayed offline (second terminal, while a model is responding)**\n\n```\nss -tlnp | grep 11434\nsudo lsof -i -nP | grep -i ollama\n```\n\nGoose, MCP, AGENTS.md, and agentgateway are all hosted by the [Agentic AI Foundation](https://aaif.io/) under the Linux Foundation.", "url": "https://wpnews.pro/news/running-goose-fully-offline-on-a-dgx-spark", "canonical_source": "https://aaif.io/blog/running-goose-fully-offline-on-a-dgx-spark/", "published_at": "2026-07-17 15:05:06+00:00", "updated_at": "2026-07-17 15:35:18.966784+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "ai-tools", "ai-products", "developer-tools"], "entities": ["Goose", "NVIDIA DGX Spark", "Ollama", "Agentic AI Foundation", "Linux Foundation", "Block", "NVIDIA GB10", "qwen3.5:35b-a3b"], "alternates": {"html": "https://wpnews.pro/news/running-goose-fully-offline-on-a-dgx-spark", "markdown": "https://wpnews.pro/news/running-goose-fully-offline-on-a-dgx-spark.md", "text": "https://wpnews.pro/news/running-goose-fully-offline-on-a-dgx-spark.txt", "jsonld": "https://wpnews.pro/news/running-goose-fully-offline-on-a-dgx-spark.jsonld"}}