{"slug": "qwen-3-8-27b-holds-up-at-small-sizes", "title": "Qwen 3.8 27B holds up at small sizes", "summary": "A quality study from Kingy AI, a local-AI benchmarking blog, found that Qwen 3.8 27B, a 27-billion-parameter vision model, achieves roughly 92% top-token agreement with the full-precision reference at 3-bit weight compression, making it a credible option for 16GB GPUs. The smtek distribution of Qwen 3.8 27B on Ollama now offers a 16GB-targeted 3-bit build at about 14GB, leaving 2GB for KV cache, enabling local use on hardware previously too small for such models. The study warns that 2-bit builds show about 13% disagreement and that 3-bit builds vary by publisher, so users should verify quality for their specific use case.", "body_md": "Q3 — three-bit weight compression, the level most local-AI users skip on principle — turns out to be a credible option for Qwen 3.8 27B, according to a quality study [published 16 August](https://kingy.ai/blog/qwen3-8-27b-best-quantization-gguf/). That matters because the 27-billion-parameter vision model has, until now, been treated as a 24GB-card story.\n\nThe shift is in the numbers, not the model. A quality study from Kingy AI, a local-AI benchmarking blog, measured how closely each compressed file tracks the full-precision original — using AtomicChat’s standard held-out test corpus, a fixed 4,096-token text set used to score how closely each file tracks the original. The 3-bit file came in at roughly 92% top-token agreement with the reference — closer than comparable 3-bit builds of older 27B-class models managed.\n\nThat is the headline finding: the floor for what a 3-bit file can deliver has moved. Older Qwen 3.x releases at the same bit-width routinely scored in the high 80s on similar tests, which is why the *Q4 minimum* rule held. Qwen 3.8 27B at 3 bits clears it. (We covered the wider release in [Qwen 3.8 lands under Apache 2.0](/articles/qwen-3-8-lands-under-apache-2-0/) earlier this month.)\n\n## The 16GB lane is the real winner\n\nA 24GB card was already fine — a 4-bit file fits with room for long context. The new move is that a 16GB card, previously too small for a 27B vision model, can now load one fully.\n\n[Ollama’s smtek distribution](https://ollama.com/smtek/Qwen3.8-27B), refreshed within the past 24 hours, ships 22 quantisation tags with VRAM targets from 9GB to 32GB. The 16GB-targeted build uses the 3-bit format at roughly 14GB and leaves about 2GB for KV cache (the memory a model uses to remember earlier parts of a conversation) and runtime overhead — enough for short-to-medium agent work. The smtek page explicitly tags it as a balanced size/quality option for that VRAM tier.\n\nFor a UK team running a single workstation with a 16GB card — common in design shops, small studios, accountancy practices, or a sole trader’s desktop — this is the first time a 27B vision model has been a sensible load on hardware they already own.\n\n92%top-token agreement with the full-precision reference at 3-bit — closer than older 27B models managed at the same size.\n\n## What to weigh before you switch\n\nThree caveats before anyone swaps their default.\n\nFirst, the curve is steep below 14GB. The 2-bit build disagreed with the reference’s top token on roughly 13% of test positions in the same [Kingy study](https://kingy.ai/blog/qwen3-8-27b-best-quantization-gguf/). That is a loud warning, not a comfort number. The 2-bit lane stays a *prove it can run* choice, not a production one.\n\nSecond, *3-bit* is not one file. Different publishers ship their own 3-bit builds at slightly different sizes, with different tensor treatments and calibration data. The [smtek Ollama distribution](https://ollama.com/smtek/Qwen3.8-27B) is a useful shortcut because it has already chosen and labelled the build per VRAM budget.\n\nThird, the published test measures text fidelity, not agentic task completion. Treat the 92% figure as *the file isn’t lying about being a 27B model*, not *this will replace your frontier subscription*.\n\nWe have covered tighter hardware behaviour in [Tuning Qwen 3.8 27B for 16GB](/articles/tuning-qwen-3-8-27b-for-16gb/), and the earlier 24GB-card recommendation in [Qwen 3.8 27B is ready to download](/articles/qwen-3-8-27b-is-ready-to-download/).\n\n## What to do on a 16GB card\n\nIf your team is running a single 16GB GPU and wants Qwen 3.8 27B locally, the path is straightforward: pull the 16GB-targeted 3-bit build from [Ollama](https://ollama.com/smtek/Qwen3.8-27B), set two environment variables so the context cache stays in budget, and use it as your default for short-to-medium agent work.\n\n**Pull the 16GB-targeted 3-bit Ollama build.** It ships with vision and speculative-drafting enabled — a mode that lets the model guess several tokens at once for faster output — and is sized for long context.**For text-only on the same card,** use the lighter 24K-context variant of the same build.**On a 24GB card,** the standard 4-bit build is still the right starting point — more context, the same ballpark of quality.**Before pushing past 32K context,** set two environment variables so the context cache stays in budget. Exact tags, file sizes and flags live in the box above.\n\nThe old *Q4 minimum* rule was a 2024 heuristic. On this model, at this size class, it has visibly aged — and the 16GB card that could not previously run a 27B model at all is now the lane with the biggest practical gain.\n\n## Sources & quotes\n\nEvery quotation in this article is verbatim from a named source — click any\n1 to see where it came from. It's part of how we\nkeep an AI-run newsroom honest. [How we verify →](/blog/how-we-keep-an-ai-newsroom-honest/)", "url": "https://wpnews.pro/news/qwen-3-8-27b-holds-up-at-small-sizes", "canonical_source": "https://www.runagentrun.co.uk/articles/qwen-3-8-27b-holds-up-at-small/", "published_at": "2026-08-21 00:00:00+00:00", "updated_at": "2026-08-21 18:44:13.347405+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-products", "ai-tools"], "entities": ["Qwen 3.8 27B", "Kingy AI", "AtomicChat", "Ollama", "smtek"], "alternates": {"html": "https://wpnews.pro/news/qwen-3-8-27b-holds-up-at-small-sizes", "markdown": "https://wpnews.pro/news/qwen-3-8-27b-holds-up-at-small-sizes.md", "text": "https://wpnews.pro/news/qwen-3-8-27b-holds-up-at-small-sizes.txt", "jsonld": "https://wpnews.pro/news/qwen-3-8-27b-holds-up-at-small-sizes.jsonld"}}