# How to Run Qwen 3 27B Locally with DeepSeek Harness > Source: > Published: 2026-08-20 00:00:00+00:00 # How to Run Qwen 3 27B Locally with DeepSeek Harness Set up Qwen 3 27B on your own hardware with DeepSeek Harness. Covers quantization, MLX vs VLM, reasoning effort, and vision capabilities. ## What is Qwen 3 27B and why run it locally? Qwen 3 27B is a dense open-weight model from Alibaba’s Qwen team that has become one of the strongest options available for running on consumer or prosumer hardware. It’s a multimodal model, meaning it handles both text and images, and it supports configurable reasoning effort, letting you trade speed for output quality on a per-task basis. On the Artificial Intelligence agentic index, it scores around 51, putting it just behind Kimi 3, a considerably larger 2.8 trillion parameter model. For a model you can realistically run on a workstation, that’s a notable result, and hands-on testing suggests the real-world performance holds up, not just the benchmark numbers. ## TL;DR **Qwen 3 27B** delivers agentic and coding performance close to Claude 4.5 on max settings from a few months ago, according to benchmark comparisons, despite running on local hardware.**Hardware choice determines your toolchain**: Apple Silicon users should grab the MLX build, while Linux and Windows users should look at VLM or SGLang for serving.** Quantization matters for throughput**: running the NVFP4 version on a two-node NVIDIA DGX Spark cluster produced roughly 15 to 20 tokens per second on a single thread, scaling to 60 to 70 tokens per second under concurrency.**Reasoning effort has four levels**(off, low, medium, extra high, with no “high” tier), and cranking it up can drastically inflate token usage and generation time without a guaranteed quality gain.**Vision capabilities are unexpectedly strong**: the model can count objects in images with patch-based reasoning and generate bounding box coordinates directly, without calling an external object-detection tool.**DeepSeek Harness adds full observability**, showing every system prompt, tool call, payload, and timing step in an auditable trajectory view, which makes debugging agentic workflows far easier than with most local setups.**The harness itself is a fast-moving open-source project**, reportedly crossing tens of thousands of GitHub stars within its first week, though the DeepSeek team is not currently accepting external contributions. ## Seven tools to build an app. Or just Remy. Editor, preview, AI agents, deploy — all in one tab. Nothing to install. ## How do you get Qwen 3 27B running on your machine? The model weights are available on Hugging Face, and which build you download depends on your hardware: **Apple Silicon**: use the MLX version, which is optimized for Apple’s unified memory architecture and tends to run more efficiently than generic builds.**Linux or Windows with NVIDIA GPUs**: VLM (vLLM) and SGLang are the two serving frameworks worth testing. Both support high-throughput inference and are commonly used for self-hosted LLM deployments.**Multi-node or higher-end setups**: quantized formats like NVFP4 let you run the full 27B dense model across clustered hardware, such as two NVIDIA DGX Spark nodes, with server access available over the local network. A single command can spin up the server, automatically downloading the model along with a smaller drafter model that enables speculative decoding, a technique where a lightweight model proposes tokens that the larger model verifies, speeding up generation without sacrificing quality. ## What hardware do you actually need? DGX Spark units are not purpose-built for a dense 27B model, so performance there gives a realistic lower bound for prosumer-tier hardware. On a two-node DGX Spark cluster running the NVFP4 quantized build, single-thread inference lands around 15 to 20 tokens per second. Under concurrent requests, aggregate throughput climbs to roughly 60 to 70 tokens per second, which is usable for agentic coding work where multiple tool calls and reasoning steps happen in parallel. If you’re on a single high-VRAM GPU or an Apple Silicon Mac with substantial unified memory, expect different numbers, but the general pattern holds: quantization and concurrent request handling matter more than raw clock speed for making a 27B dense model feel responsive. ## How do you configure Qwen 3 27B inside DeepSeek Harness? DeepSeek Harness is built around a plugin architecture, where every capability, from model providers to tools, is modular. To add Qwen 3 27B as a provider: - Open the harness settings. - Navigate to the model configuration option, which opens a YAML file. - Define your provider entry, pointing to your running Qwen instance (for example, the MLX version locally, or a networked endpoint if you’re serving from a DGX cluster or another machine). - Save and select the new configuration as your active model. This setup lets you swap between multiple models, including other DeepSeek models like a V4 flash variant, without restructuring your workflow. Running the model across two DGX Spark nodes, for instance, was specifically chosen to preserve a full context window while handling multiple concurrent requests, something a single node couldn’t sustain at the same capacity. ## What reasoning effort setting should you use? Qwen 3 27B exposes four reasoning effort levels: completely off, low, medium, and extra high (there is no “high” tier). This setting has an outsized effect on both output quality and resource consumption. With reasoning off, the model behaves like a standard non-reasoning model. In one test generating a self-descriptive website using web search, output landed around 20,000 tokens and produced noticeably weaker, “AI slop” style results with fabricated numbers. ## Remy doesn't build the plumbing. It inherits it. Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something. Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want. Switching to low reasoning produced a substantially better result from the same prompt, a clean, usable website. Medium reasoning added polish, like animations, but also consumed far more tokens, with one run reaching around 860,000 input tokens and roughly 63 minutes of generation time. Extra high reasoning delivered the best visual quality and consistency, but at the cost of even higher runtime and token usage, sometimes near 500,000 input tokens and over 80 minutes for a single task. The takeaway: higher reasoning effort doesn’t scale linearly with quality, and it can eat your entire completion budget as “thinking” without producing final output at all. Match the setting to the task. Quick iterations favor low reasoning; final-quality generation may justify extra high, but budget for significantly longer wait times. ## Is Qwen 3 27B actually good at vision tasks? Yes, and notably better than expected for a model this size. Vision language models are typically strong at describing images but weak at counting objects within them. Qwen 3 27B pushed back against that pattern in testing: asked to count cars in a complex parking lot image, it broke the scene into regions (a top row, a main lot, and additional areas), counted objects within each region, and arrived at a total that was reasonably accurate when checked against the actual image. It also generated bounding box coordinates for detected objects directly, without invoking any external object-detection model. The coordinates weren’t perfect, some boxes were oversized or slightly misaligned, but the overall detection and placement were accurate across multiple object types and orientations. This kind of native spatial reasoning, without tool-assisted detection, is uncommon in open vision-language models. ## Why does the DeepSeek Harness trajectory view matter? Beyond running the model, DeepSeek Harness offers a “trajectory” feature that logs the entire execution path of an agentic session: the system prompt, the user message, the full context the model had access to, every tool call with its exact payload and schema, the results returned, and timing data for each step. This level of built-in observability is unusual. Most local agent setups require bolting on a separate tracing or observability tool to get this kind of visibility. With it built in, you can audit exactly where an agent’s reasoning went wrong, identify which tool calls are failing repeatedly, and adjust your plugin configuration accordingly, all without leaving the harness interface. ## Frequently Asked Questions ### What quantization format should I use for Qwen 3 27B? NVFP4 is a viable option for running the full dense model with reduced memory footprint on NVIDIA hardware, as demonstrated on a two-node DGX Spark cluster. Exact format availability depends on what’s published on Hugging Face for your specific hardware target, so check the model repository for MLX, GGUF, or FP4 variants that match your setup. ### Does Qwen 3 27B beat larger models like Kimi 3? Not quite. On the Artificial Intelligence agentic index, Qwen 3 27B scores just behind Kimi 3, a 2.8 trillion parameter model, despite being dramatically smaller. It’s not outright better, but the gap is small relative to the size difference. ### Why does increasing reasoning effort sometimes produce worse or no output? Higher reasoning settings allocate more of the token budget to internal “thinking” before the model produces a final answer. In some cases, especially with medium or extra high settings, the model can consume the entire completion budget on reasoning and leave nothing for the final output, resulting in no usable response. ### Everyone else built a construction worker. We built the contractor. One file at a time. UI, API, database, deploy. ### Can Qwen 3 27B draw bounding boxes without extra tools? Yes. In testing, the model generated bounding box coordinates for detected objects directly from its own visual analysis, then used those coordinates in a Python script to draw the boxes. It didn’t rely on a separate object-detection model, which is uncommon for open vision-language models of this size. ### What’s the difference between MLX, VLM, and SGLang for serving this model? MLX is optimized specifically for Apple Silicon and its unified memory architecture. VLM (vLLM) and SGLang are both high-throughput serving frameworks designed for NVIDIA GPUs on Linux or Windows, each with different optimization strategies for batching and request concurrency.