{"slug": "qwen-3-8-27b-at-2x-speed-on-a-5090", "title": "Qwen 3.8 27B at 2x speed on a 5090", "summary": "Adore LLC released Balto Speedrunner, a Windows app that turns Qwen 3.8 27B into a local coding agent for a single RTX 5090, achieving up to 300 tok/s on chat prompts and 150 tok/s during coding runs. The installer requires at least 90 GB free disk space and uses Docker Desktop, SGLang, and RadixArk models. Balto supports Tailscale Serve for private remote access and is proprietary software copyright 2026 Adore LLC.", "body_md": "On a Mac? Get Balto for Mac\n\nFirst launch downloads the inference engine and model. Keep at least 90 GB free.\n\n[ How it works](#what-the-installer-does)\n•\n\n[•](#measured-on-our-rtx-5090)\n\n**Benchmarks**\n\n**Support Balto** Balto turns Qwen 3.8 27B into a fast local coding agent for one RTX 5090. Expect roughly 150 tok/s during real coding runs and up to 300 tok/s on clean chat prompts.\n\nRun the installer, approve Windows if it asks to enable WSL, and Balto handles the rest. It resumes setup after a required restart, preserves partial downloads, and opens the coding workspace as soon as the model is ready.\n\nThe Windows app uses Tauri and the system WebView2 runtime. Model inference runs in Docker, and Balto keeps its Node.js workspace runtime in its own app data directory.\n\n| Workload | Output speed | Context | Notes |\n|---|---|---|---|\n| Chat | Up to 300 tok/s | Short prompt | Warm model |\n| Code | 150 tok/s | Live agent session | Real tool calls |\n\n- Confirms that the PC has one RTX 5090, enough free disk space, and current NVIDIA support.\n- Installs Docker Desktop in its official per-user WSL 2 mode, or starts an existing installation.\n- Downloads the\n`lmsysorg/sglang:qwen38-27b`\n\nimage at the exact tested digest. - Creates a persistent Docker volume for model files, so interrupted downloads resume and updates do not erase weights.\n- Starts\n`RadixArk/Qwen3.8-27B-NVFP4`\n\nwith`RadixArk/Qwen3.8-27B-DSpark`\n\nusing the tested 80K configuration. - Installs the coding workspace into Balto's private app directory.\n- Reminds the owner to unload other local models before starting Balto.\n- Applies future performance configuration updates without deleting the persistent model cache.\n\nFirst launch requires a large download and at least 90 GB of free disk space.\n\nBalto can use Tailscale Serve after the owner signs in to Tailscale. It keeps the workspace bound to `127.0.0.1`\n\nand exposes private HTTPS endpoints only inside the user's tailnet. Balto does not enable Tailscale Funnel or open a public router port.\n\nThe onboarding screen shows the exact private URL and lets the owner turn remote access off without changing unrelated Tailscale routes.\n\nThe inference arguments live in [ runtime/balto.ps1](/jtc268/balto-speedrunner/blob/main/runtime/balto.ps1). The important settings are:\n\n```\nmodel                  Qwen 3.8 27B NVFP4\ncontext length         80000\nattention backend      flashinfer\nmax running requests   1\nspeculation            DSpark, FP8 draft\nsampling               temperature 0.6, top_p 0.95, top_k 20\n```\n\nBalto uses safe sampling defaults for coding. It does not force greedy temperature zero sampling, which can trap this model in repetitive reasoning loops.\n\nRequirements:\n\n- Windows 11\n- Node.js 22 or newer\n- Rust stable with the MSVC target\n- WebView2\n\n```\nnpm install\nnpm run check\nnpm run dev\n```\n\nBuild the NSIS installer:\n\n```\nnpm run build\n```\n\nBalto supports two different signatures:\n\n- Tauri updater signatures protect update artifacts and are required by the in-app updater.\n- Windows Authenticode identifies Adore LLC as the publisher and prevents the unsigned-app SmartScreen warning.\n\nEvery install shows its version in Settings. A green update arrow appears when GitHub publishes a newer signed release; one click verifies, installs, and relaunches it.\n\nThe release workflow uses Azure Artifact Signing when publisher credentials are configured. The in-app updater always verifies Tauri update signatures. Local development builds remain unsigned.\n\nBalto Speedrunner is proprietary software, copyright 2026 Adore LLC. All rights reserved.\n\nThe coding agent interface integrates MIT-licensed software from DeepSeek AI. Inference is powered by SGLang. Qwen model weights remain under their own license. See [ THIRD_PARTY_NOTICES.md](/jtc268/balto-speedrunner/blob/main/THIRD_PARTY_NOTICES.md) for the full notices.\n\nBalto Speedrunner is not affiliated with or endorsed by DeepSeek, Qwen, Alibaba, SGLang, LMSYS, NVIDIA, Docker, Tailscale, Microsoft, or OpenAI.\n\nIf Balto saves you setup time, [buy me a coffee](https://buymeacoffee.com/refresh1).", "url": "https://wpnews.pro/news/qwen-3-8-27b-at-2x-speed-on-a-5090", "canonical_source": "https://github.com/jtc268/balto-speedrunner", "published_at": "2026-08-15 16:21:13+00:00", "updated_at": "2026-08-15 16:40:48.093965+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools"], "entities": ["Adore LLC", "Balto Speedrunner", "Qwen 3.8 27B", "RTX 5090", "Docker Desktop", "SGLang", "RadixArk", "Tailscale"], "alternates": {"html": "https://wpnews.pro/news/qwen-3-8-27b-at-2x-speed-on-a-5090", "markdown": "https://wpnews.pro/news/qwen-3-8-27b-at-2x-speed-on-a-5090.md", "text": "https://wpnews.pro/news/qwen-3-8-27b-at-2x-speed-on-a-5090.txt", "jsonld": "https://wpnews.pro/news/qwen-3-8-27b-at-2x-speed-on-a-5090.jsonld"}}