{"slug": "the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026", "title": "The Best Open LLM You Can Actually Run Right Now, by VRAM Tier (August 2026)", "summary": "As of August 2026, the best open LLM to run locally depends on VRAM, with gpt-oss-20b recommended for 8-12GB, Gemma 4 31B or Qwen3.6-35B-A3B for 24GB, and gpt-oss-120b or DeepSeek V4 Flash for 128GB unified memory, according to Vetted Consumer's synthesis of vendor specs, benchmarks, and owner reports. The guide notes that Mixture-of-Experts models now make 30B-class models outperform larger ones, and recommends matching the model to memory rather than seeking the largest size.", "body_md": "**The short answer, August 2026:** match the model to your memory. On 8 to 12GB of VRAM, run gpt-oss-20b. On 24GB (an RTX 3090 or 4090), run Gemma 4 31B or Qwen3.6-35B-A3B. On a 128GB unified box, run gpt-oss-120b or DeepSeek V4 Flash. Above that, GLM-5.2 or Inkling; for Kimi K3, rent. The table and the reasons are below, and if you want the exact quant for your machine, our [Can I run it? calculator](https://vettedconsumer.com/can-i-run-it/) answers it directly.\n\nTwo things make this a genuinely different question than it was six months ago. The models all went [Mixture-of-Experts](https://vettedconsumer.com/every-frontier-open-model-is-a-moe-now-what-that-does-to-your-hardware-math/), so a 30B model now punches like a much bigger one, and a wave of small models got so good that, as one r/LocalLLaMA owner put it, \"the current gen models that are around 25-35b are so good that they knocked out anything above them until you get into a couple hundred b parameters.\" So the map is not \"bigger is better.\" It is \"run the best model your memory holds, and stop there.\" We have not benchmarked every model first-hand; this synthesizes vendor specs, third-party benchmarks, and owner reports, all cited.\n\n## The quick-reference table\n\n| Your hardware | Sweet-spot pick (Aug 2026) | Also strong |\n|---|---|---|\n| 8 to 12GB VRAM | gpt-oss-20b (Apache 2.0, ~12GB) | Gemma 4 12B, Ministral-3 8B, Granite 4.1 |\n| 16GB VRAM | Qwen3-30B-A3B (Q4 MoE) | gpt-oss-20b full, Gemma 4 26B-A4B |\n| 24GB (RTX 3090 / 4090) | Gemma 4 31B or Qwen3.6-35B-A3B | gpt-oss-20b, any 30B-class MoE |\n| 128GB unified (Strix Halo / Mac) | gpt-oss-120b or DeepSeek V4 Flash | Qwen 3.6 35B, Gemma 4 31B + MTP |\n| 512GB+ (Mac Studio / cluster) | GLM-5.2 or Inkling (2 to 3-bit) | rent a GPU node for Kimi K3 |\n| Best for coding, any tier | DeepSeek V4 Flash | Qwen3-Coder-Next, GLM-5.2 |\n\n## Best local LLM for 8 to 12GB VRAM\n\nThis is most gaming PCs (an RTX 3060, 4060, or 5060). The pick is **gpt-oss-20b**: OpenAI's small open model is a 21B Mixture-of-Experts with only 5.1B active parameters, Apache 2.0, and it ships in MXFP4 at roughly 12GB, so it is 4-bit at its native precision rather than a lossy afterthought. It reasons well above its size and runs fast because so little of it fires per token; owners report 50 to 60+ tokens per second on capable hardware. For a dense alternative with strong writing and emotional range, **Gemma 4 12B** is the other pick, and **Ministral-3 8B** or **Granite 4.1** cover the very tight cases. If you have 12GB of VRAM plus a healthy pile of system RAM, you can also reach up a tier with expert offload (more on that below).\n\n## Best local LLM for 16GB VRAM\n\n16GB (an RTX 4060 Ti 16GB, 5060 Ti 16GB, or Arc B580) opens the door to the 30B MoE class. **Qwen3-30B-A3B** at Q4 is the value pick: 30B total, 3B active, so it decodes at small-model speed while holding mid-model knowledge. gpt-oss-20b now runs with full context headroom, and Gemma 4's 26B-A4B MoE variant fits comfortably. This is the tier where the \"a good 30B beats everything up to a few hundred B\" rule starts paying off, because these models fit and fly.\n\n## Best local LLM for 24GB (RTX 3090 or 4090)\n\nThe enthusiast sweet spot. A used [RTX 3090](https://vettedconsumer.com/used-rtx-3090-2026-local-ai-best-deal/) or a 4090 gives you 24GB at roughly 936 GB/s of bandwidth, which is fast enough that mid-size models feel instant. Two picks split by use. For conversation, reasoning, and general smarts, **Gemma 4 31B** is the community favorite; in the \"best chat model\" threads on [r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/comments/1v6ig61/best_chat_model_that_fits_in_128gb/?ref=vettedconsumer.com), owners repeatedly land on it: \"the best conversationalist, is quite smart, and has good emotional IQ compared to other, even bigger models.\" Run it at a Q4-class quant, and add its [MTP draft model for a free speed boost](https://vettedconsumer.com/speculative-decoding-explained-the-free-speed-toggle-your-local-llm-is-probably-not-using/). For agentic and mixed work, **Qwen3.6-35B-A3B** is the sparse-MoE alternative that decodes faster. Either way, 24GB is where a single card stops feeling like a compromise.\n\n## Best local LLM for 128GB unified memory (Strix Halo, Mac, DGX Spark)\n\nThe 128GB tier (a [Strix Halo mini-PC](https://vettedconsumer.com/gmktec-evo-x3-tested-by-wendell-the-fastest-strix-halo-yet-and-its-unholy-oculink-trick/), a Mac, or a DGX Spark) is the one that reaches frontier-adjacent models. Two picks, by job. For a fast, capable all-rounder, **gpt-oss-120b**: the 117B MoE runs at a measured 56 tokens per second on a $1,900 Strix box, plenty for real work. For agentic coding and million-token context, **DeepSeek V4 Flash**: the 284B MoE scores one point behind the 753B GLM-5.2 on the independent Artificial Analysis index at a third the size, and owners fit its dynamic quants on a single 128GB machine, [as we covered in full](https://vettedconsumer.com/deepseek-v4-flash-tested-frontier-class-coding-for-79-cents-a-day-and-it-runs-on-a-128gb-box/). The 128GB \"best fit\" threads bear this out, with owners naming DeepSeek V4 Flash for agentic use and Gemma 4 31B for pure chat. This is the tier where \"run it locally\" and \"run something genuinely strong\" finally overlap.\n\n## Best local LLM for 512GB and up\n\nAt 512GB (a maxed Mac Studio) or a small cluster, the true giants come into range, though \"into range\" still means aggressive quants. **GLM-5.2** (753B) and **Inkling** (975B) both fit a single 512GB Mac at a 2-to-3-bit build, the [reversal Inkling made possible](https://vettedconsumer.com/inkling-mira-muratis-first-open-model-is-a-975b-moe-you-can-actually-run/). The one model that still does not fit any single box is [Kimi K3](https://vettedconsumer.com/kimi-k3-the-largest-open-model-ever-2-8t-params-and-why-almost-no-one-can-run-it-locally/) (2.8T, 104B active): its smallest usable quant is 594GB, so it wants a cluster or, for almost everyone, a rented GPU node. That buy-vs-rent line is exactly what our [cost calculator](https://vettedconsumer.com/cost-calculator/) is built to price, and for a model you touch occasionally, renting usually wins.\n\n## Best local LLM for coding\n\nCoding is the one workload where the pick changes, because agentic benchmarks reward different models. **DeepSeek V4 Flash** is the standout: MIT-licensed, 284B total, and it jumped to 54.4 on the DeepSWE agentic benchmark (a DeepSeek-reported figure), which is what put it on every coder's radar this month. If you want a smaller Apache-licensed coder, **Qwen3-Coder-Next** (80B total, 3B active) is the one owners say they can genuinely run. Above those, **GLM-5.2** leads several coding suites but demands the 512GB tier. Whichever you pick, pair it with a [draft model](https://vettedconsumer.com/speculative-decoding-explained-the-free-speed-toggle-your-local-llm-is-probably-not-using/) for a real speedup on the deterministic text that code generation is full of.\n\n## The one trick that moves you up a tier\n\nYou are not locked to your VRAM. Because these models are sparse MoE, [llama.cpp's expert offload](https://vettedconsumer.com/every-frontier-open-model-is-a-moe-now-what-that-does-to-your-hardware-math/) (`--n-cpu-moe`\n\n) keeps the rarely-touched expert weights in cheap system RAM while the hot parts stay on the GPU. A 24GB card plus 128GB of system RAM can run models that would never fit in VRAM alone: slower than unified memory, far cheaper, and it fits. That single flag is why \"what fits in my VRAM\" and \"what I can run\" are no longer the same question.\n\n## How to pick for your exact machine\n\nThe tiers above are the map; your machine is the specifics. Bandwidth sets how fast any of these decode ([why here](https://vettedconsumer.com/bandwidth-not-tflops-what-sets-your-local-llm-speed-and-why-the-newest-card-isnt-always-fastest/)), and the quant you pick sets whether it fits at all ([the quant guide](https://vettedconsumer.com/gguf-vs-gptq-vs-awq-the-plain-english-guide-to-llm-quantization-and-which-one-to-pick/)). Rather than guess, put your GPU or box into our [Can I run it? tool](https://vettedconsumer.com/can-i-run-it/): it returns the models that fit, the largest quant that fits, and the theoretical tokens per second, and the [quant picker](https://vettedconsumer.com/quant-picker/) tells you the exact file to download. That is the fastest way from \"what should I run\" to a model actually loaded on your hardware.\n\n## Sources and how we researched this\n\n- Model specs and licenses: vendor model cards and our per-model coverage, linked inline (\n[DeepSeek V4 Flash](https://vettedconsumer.com/deepseek-v4-flash-tested-frontier-class-coding-for-79-cents-a-day-and-it-runs-on-a-128gb-box/),[Inkling](https://vettedconsumer.com/inkling-mira-muratis-first-open-model-is-a-975b-moe-you-can-actually-run/),[Kimi K3](https://vettedconsumer.com/kimi-k3-the-largest-open-model-ever-2-8t-params-and-why-almost-no-one-can-run-it-locally/),[GLM-5.2](https://vettedconsumer.com/glm-5-2-the-most-powerful-open-weight-model-yet-and-the-brutal-reality-of-running-it-locally/)). gpt-oss figures from OpenAI's release; the DeepSWE score is DeepSeek's own reported number. - Third-party throughput: the measured gpt-oss-120b Strix Halo figure from the community benchmark grid cited in our\n[Strix Halo coverage](https://vettedconsumer.com/gmktec-evo-x3-tested-by-wendell-the-fastest-strix-halo-yet-and-its-unholy-oculink-trick/); the independent Artificial Analysis index for cross-model ranking. - Owner consensus: the r/LocalLLaMA\n[\"best chat model that fits in 128gb\"](https://www.reddit.com/r/LocalLLaMA/comments/1v6ig61/best_chat_model_that_fits_in_128gb/?ref=vettedconsumer.com)thread and related discussions, quoted verbatim and attributed. - This is a synthesis of vendor documentation, independent benchmarks, and owner reports as of August 2026; we have not tested every model first-hand, and the picks will move as new models ship. Prices and quants shift weekly in the current memory market.\n\n*Related: **Every frontier open model is a MoE now** · **What \"open weights\" actually lets you do** · **The local-LLM hardware cheat sheet** · **Speculative decoding, explained*", "url": "https://wpnews.pro/news/the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026", "canonical_source": "https://vettedconsumer.com/best-open-llm-to-run-by-vram-tier-2026/", "published_at": "2026-08-24 00:28:01+00:00", "updated_at": "2026-08-24 00:43:41.757204+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "generative-ai", "ai-products", "ai-tools"], "entities": ["OpenAI", "gpt-oss-20b", "Gemma 4 31B", "Qwen3.6-35B-A3B", "DeepSeek V4 Flash", "GLM-5.2", "Inkling", "Kimi K3"], "alternates": {"html": "https://wpnews.pro/news/the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026", "markdown": "https://wpnews.pro/news/the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026.md", "text": "https://wpnews.pro/news/the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026.txt", "jsonld": "https://wpnews.pro/news/the-best-open-llm-you-can-actually-run-right-now-by-vram-tier-august-2026.jsonld"}}