{"slug": "how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac", "title": "How to (easily) run a fully local, private AI assistant with OpenClaw and QVAC", "summary": "OpenClaw, the most widely used agent harness, can now be paired with QVAC, an open-source SDK from Tether, to run a fully local, private AI assistant. The setup uses QVAC's OpenAI-compatible server to connect OpenClaw to a local model like Qwen3-8B, keeping all inference on-device and eliminating reliance on cloud services.", "body_md": "A language model on its own can only produce text. It cannot open a file, run a command, call an API, or remember what it did five minutes ago. An agent harness is the layer that closes that gap. It takes the model's text output, turns it into real actions (read this file, run this command, search this folder), feeds the results back to the model, and loops until the task is done. The model is the brain. The harness is the hands.\n\nOpenClaw is one of these harnesses, and right now it is the most widely used one. It crossed a large install base in a few months and sits at the top of agent usage charts. You give it a goal in plain language, and it plans, calls tools, and reports back. It speaks to any model that exposes an OpenAI-compatible API, which is the detail that matters for this guide: you are not locked to a single cloud vendor. You point it at whatever model endpoint you want, including one running on your own machine.\n\nThat is the whole idea here. OpenClaw for the agent loop, QVAC for the model, both on hardware you control.\n\n// Detect dark theme var iframe = document.getElementById('tweet-2067624599270166813-984'); if (document.body.className.includes('dark-theme')) { iframe.src = \"https://platform.twitter.com/embed/Tweet.html?id=2067624599270166813&theme=dark\" }\n\nThe harness is general, so the use cases are broad. The common ones:\n\nIn every one of these, the agent is taking actions on real files and a real system. Which is exactly why where the model runs starts to matter.\n\nWhen a model only chats, where it runs is mostly a question of privacy. When a model drives an agent that reads your files and runs commands, it becomes a question of control. The agent acts on your machine, and if the model behind it lives in someone else's datacenter, every file it reads and every command it runs depends on a system you do not own: its uptime, its latency, its terms, and its access policy, any of which can change without you. Running the model locally keeps the agent answerable to your hardware, not to a service that can throttle it, change it, or cut it off.\n\nRunning the model locally changes the property, not just the policy:\n\nThe trade is latency and raw capability: a model that fits on a laptop is smaller than a frontier cloud model, so it is slower and less sharp. For a large class of real work, that trade is worth it, and it keeps getting better as small models improve.\n\nQVAC is the piece that makes the local half practical. It is an open-source SDK and CLI from Tether that runs models on your own device across every major GPU backend (NVIDIA, AMD, Intel, Adreno (Qualcomm) and Mali on Linux, Windows and Android through Vulkan, and Apple Silicon through Metal), and it ships an OpenAI-compatible server. That server is the bridge OpenClaw plugs into.\n\nThree pieces, two of them long-running:\n\nNothing in this chain calls out to the internet for inference. The model file is downloaded once, then everything happens on the machine.\n\nThe setup uses Qwen3-8B, which is the best balance for most laptops. If you have less memory, drop to the 4B. If you have a workstation and want stronger results on multi-step coding tasks, step up to the Qwen3.6-27B multimodal model. To switch, change the model name in the config file (step 2 of the setup) to the constant in the last column.\n\n| Use case | Model | Recommended RAM | Required storage | Config name |\n|---|---|---|---|---|\n| Light and fast, mostly conversation | Qwen3-4B (4-bit) | 8 GB | ~3 GB | `QWEN3_4B_INST_Q4_K_M` |\n| Recommended default, good all-round balance | Qwen3-8B (4-bit) | 16 GB | ~6 GB | `QWEN3_8B_INST_Q4_K_M` |\n| Heavier coding, multi-step agents, and vision | Qwen3.6-27B multimodal (4-bit) | 48 GB | ~18 GB | `QWEN3_6_27B_MULTIMODAL_Q4_K_XL` |\n\nA GPU helps a lot but is not required. QVAC uses Apple Metal on Apple Silicon and Vulkan on NVIDIA, AMD, and Intel. On a CPU-only machine it still runs, just slower. Run `qvac doctor`\n\nto see what your hardware supports.\n\nThe walkthrough below is pure copy-paste. Run each step in order, and you will have a local coding agent in a few minutes. The only thing that takes real time is the first model download. The setup uses Qwen3-8B quantized at 4 bits (about 4.7 GB), which downloads once and is cached after that.\n\nThere are two paths through it, and the steps below handle both:\n\n**Requirement:** Node.js 22.17 or newer (check with `node --version`\n\n, get it from nodejs.org). About 5 GB of free disk for the model.\n\n**1. Install the QVAC CLI** (all platforms)\n\n```\nnpm install -g @qvac/cli @qvac/sdk\n```\n\n**2. Create a model config.** Makes a folder and writes one small config file.\n\nmacOS and Linux:\n\n```\nmkdir -p ~/qvac-openclaw && cd ~/qvac-openclaw\ncat > qvac.config.json <<'EOF'\n{\n  \"plugins\": [\"@qvac/sdk/llamacpp-completion/plugin\"],\n  \"serve\": {\n    \"models\": {\n      \"qwen3-8B-Q4-chat\": {\n        \"model\": \"QWEN3_8B_INST_Q4_K_M\",\n        \"type\": \"llamacpp-completion\",\n        \"preload\": true,\n        \"config\": { \"tools\": true, \"toolsMode\": \"static\", \"ctx_size\": 16384, \"gpu_layers\": -1, \"reasoning_budget\": 0 }\n      }\n    }\n  }\n}\nEOF\n```\n\nWindows (PowerShell):\n\n```\nmkdir $HOME\\qvac-openclaw; cd $HOME\\qvac-openclaw\n@'\n{\n  \"plugins\": [\"@qvac/sdk/llamacpp-completion/plugin\"],\n  \"serve\": {\n    \"models\": {\n      \"qwen3-8B-Q4-chat\": {\n        \"model\": \"QWEN3_8B_INST_Q4_K_M\",\n        \"type\": \"llamacpp-completion\",\n        \"preload\": true,\n        \"config\": { \"tools\": true, \"toolsMode\": \"static\", \"ctx_size\": 16384, \"gpu_layers\": -1, \"reasoning_budget\": 0 }\n      }\n    }\n  }\n}\n'@ | Set-Content -Encoding utf8 qvac.config.json\n```\n\n**3. Start the QVAC server** (all platforms). Run it in that folder and leave the terminal open. `qwen3-8B-Q4-chat`\n\nis the alias you defined in step 2; it serves Qwen3-8B at 4-bit (Q4_K_M). First run downloads the model once (about 4.7 GB). Ready when you see `QVAC API server listening`\n\n.\n\n```\nqvac serve openai --model qwen3-8B-Q4-chat\n```\n\n**4. Install OpenClaw (optional).** Skip if you already have it.\n\n```\nnpm install -g openclaw\n```\n\n**5. Point OpenClaw at QVAC** (new terminal). The first command connects OpenClaw to the local server. The next three keep the agent fast and reliable on a local model.\n\n```\nopenclaw onboard --auth-choice custom-api-key --custom-base-url http://127.0.0.1:11434/v1 --custom-model-id qwen3-8B-Q4-chat --custom-api-key \"qvac\" --non-interactive --accept-risk --skip-channels --skip-daemon --skip-search --skip-ui --skip-skills --skip-health\n```\n\nTwo values here tie back to step 2. `--custom-model-id qwen3-8B-Q4-chat`\n\nmust match the model alias in your config (the key under `serve.models`\n\n). The `--custom-api-key \"qvac\"`\n\nis only a placeholder: the local QVAC server does not require a key, but OpenClaw's setup needs the field filled, so any non-empty value works.\n\n```\nopenclaw config set tools.profile coding\nopenclaw config set tools.allow '[\"write\",\"read\",\"exec\"]' --strict-json\nopenclaw config set models.providers.custom-127-0-0-1-11434.timeoutSeconds 600\n```\n\n**6. Start the agent gateway** (new terminal, leave open). Ready when you see `ready`\n\n.\n\n```\nopenclaw gateway run\n```\n\n**7. Talk to your local agent** (new terminal). Try turning off your network and asking again.\n\n```\nopenclaw agent --agent main --message \"Are you running in the cloud or on my machine? Answer in one sentence.\"\n```\n\n**8. Have it build something (optional).** Asks the agent to write a small animated web page and open it. On a local model it takes a minute or two. The open command differs per platform: `open`\n\non macOS, `xdg-open`\n\non Linux, `start`\n\non Windows.\n\n```\nopenclaw agent --agent main --message \"Create an HTML file at ~/openclaw_lobster.html showing a large lobster emoji at 140px pulsing with a CSS scale animation on a dark #0f1410 background, with the text 'openclaw running on local with QVAC' in teal #16E3C1 monospace below it, fully visible immediately. Then run the shell command: open ~/openclaw_lobster.html\"\n```\n\nYou now have a coding agent running entirely on your own machine. A good first test is to confirm the obvious: ask it whether it is running in the cloud or on your machine, then turn off your network and ask again. It keeps answering, because nothing was ever being sent to a server in the first place.\n\nFor something you can see, hand it a small build task and watch it write a file and open it. A coding task like generating a small web page runs in a minute or two on a typical laptop, fully offline, because the model is doing real work locally rather than streaming from a datacenter. The setup includes a ready-made example: it asks the agent to write a tiny animated web page and open it in your browser, with no further input from you.\n\nFrom here, the same setup handles the rest of what OpenClaw does. Point it at a project folder, ask it to read and edit code, have it summarize a directory, or give it a multi-step chore. The agent loop is identical. The only thing that changed is that the intelligence behind it is yours.", "url": "https://wpnews.pro/news/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac", "canonical_source": "https://dev.to/qvac/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac-2bmp", "published_at": "2026-07-29 09:22:29+00:00", "updated_at": "2026-07-29 09:38:17.003105+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "developer-tools", "ai-infrastructure"], "entities": ["OpenClaw", "QVAC", "Tether", "Qwen3-8B", "Qwen3-4B", "Qwen3.6-27B"], "alternates": {"html": "https://wpnews.pro/news/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac", "markdown": "https://wpnews.pro/news/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac.md", "text": "https://wpnews.pro/news/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac.txt", "jsonld": "https://wpnews.pro/news/how-to-easily-run-a-fully-local-private-ai-assistant-with-openclaw-and-qvac.jsonld"}}