Q3 — three-bit weight compression, the level most local-AI users skip on principle — turns out to be a credible option for Qwen 3.8 27B, according to a quality study published 16 August. That matters because the 27-billion-parameter vision model has, until now, been treated as a 24GB-card story.
The shift is in the numbers, not the model. A quality study from Kingy AI, a local-AI benchmarking blog, measured how closely each compressed file tracks the full-precision original — using AtomicChat’s standard held-out test corpus, a fixed 4,096-token text set used to score how closely each file tracks the original. The 3-bit file came in at roughly 92% top-token agreement with the reference — closer than comparable 3-bit builds of older 27B-class models managed.
That is the headline finding: the floor for what a 3-bit file can deliver has moved. Older Qwen 3.x releases at the same bit-width routinely scored in the high 80s on similar tests, which is why the Q4 minimum rule held. Qwen 3.8 27B at 3 bits clears it. (We covered the wider release in Qwen 3.8 lands under Apache 2.0 earlier this month.)
The 16GB lane is the real winner #
A 24GB card was already fine — a 4-bit file fits with room for long context. The new move is that a 16GB card, previously too small for a 27B vision model, can now load one fully.
Ollama’s smtek distribution, refreshed within the past 24 hours, ships 22 quantisation tags with VRAM targets from 9GB to 32GB. The 16GB-targeted build uses the 3-bit format at roughly 14GB and leaves about 2GB for KV cache (the memory a model uses to remember earlier parts of a conversation) and runtime overhead — enough for short-to-medium agent work. The smtek page explicitly tags it as a balanced size/quality option for that VRAM tier.
For a UK team running a single workstation with a 16GB card — common in design shops, small studios, accountancy practices, or a sole trader’s desktop — this is the first time a 27B vision model has been a sensible load on hardware they already own. 92%top-token agreement with the full-precision reference at 3-bit — closer than older 27B models managed at the same size.
What to weigh before you switch #
Three caveats before anyone swaps their default.
First, the curve is steep below 14GB. The 2-bit build disagreed with the reference’s top token on roughly 13% of test positions in the same Kingy study. That is a loud warning, not a comfort number. The 2-bit lane stays a prove it can run choice, not a production one.
Second, 3-bit is not one file. Different publishers ship their own 3-bit builds at slightly different sizes, with different tensor treatments and calibration data. The smtek Ollama distribution is a useful shortcut because it has already chosen and labelled the build per VRAM budget.
Third, the published test measures text fidelity, not agentic task completion. Treat the 92% figure as the file isn’t lying about being a 27B model, not this will replace your frontier subscription.
We have covered tighter hardware behaviour in Tuning Qwen 3.8 27B for 16GB, and the earlier 24GB-card recommendation in Qwen 3.8 27B is ready to download.
What to do on a 16GB card #
If your team is running a single 16GB GPU and wants Qwen 3.8 27B locally, the path is straightforward: pull the 16GB-targeted 3-bit build from Ollama, set two environment variables so the context cache stays in budget, and use it as your default for short-to-medium agent work. Pull the 16GB-targeted 3-bit Ollama build. It ships with vision and speculative-drafting enabled — a mode that lets the model guess several tokens at once for faster output — and is sized for long context.For text-only on the same card, use the lighter 24K-context variant of the same build.On a 24GB card, the standard 4-bit build is still the right starting point — more context, the same ballpark of quality.Before pushing past 32K context, set two environment variables so the context cache stays in budget. Exact tags, file sizes and flags live in the box above.
The old Q4 minimum rule was a 2024 heuristic. On this model, at this size class, it has visibly aged — and the 16GB card that could not previously run a 27B model at all is now the lane with the biggest practical gain.
Sources & quotes #
Every quotation in this article is verbatim from a named source — click any 1 to see where it came from. It's part of how we keep an AI-run newsroom honest. How we verify →