Three 30-billion-parameter class models sit on your shortlist: Muse Glimmer, Gemma 4 and Qwen 3.6. Each vendor’s benchmark table names a different winner, and there’s little like-for-like guidance on which one fits a local-inference strategy. That makes the choice feel like a bet on marketing numbers.
This article is a decision-oriented read: what each model is, which workload each suits, and the licence, VRAM and inference mechanics that decide whether a 30B model belongs in your stack. Guessing wrong means a model that won’t fit your GPU, a licence you can’t live with, or the data-residency compromises you set out to avoid.
What is Muse Glimmer, and how does it differ from Meta’s earlier Llama models? #
Muse Glimmer is a 30-billion-parameter model from Meta Superintelligence Labs, released in August 2026 under an Apache 2.0 licence and built to run on a single consumer GPU. It’s a dense model, so all 30B parameters fire on every pass.
Muse Glimmer breaks from the Llama lineage in three ways. First, the naming: a new “Muse” family rather than a continuation of Llama. Second, the licence: Llama 4 ships under the Llama Community Licence, while Muse Glimmer ships under Apache 2.0 (the full licence story is worth a separate read). Third, the design target: local-first, single-GPU inference rather than an afterthought.
Glimmer is distilled from Muse Spark 1.2, and it sits in the same release wave as Muse Code, Meta’s terminal-based coding agent. Muse Code’s 70.6% on Meta’s internal coding benchmark, against 65.4% for GPT-5.6 Terra, signals the family’s ambition. A January 2026 knowledge cutoff means anything newer needs retrieval or tools.
Muse Glimmer vs Gemma 4 vs Qwen: which open model should you choose? #
There is no single best model here. The right pick depends on whether you need local, private agent inference, cloud serving, or fine-tuning. Treat any vendor benchmark table as directional until you’ve validated it on your own workload.
Muse Glimmer is a 30B dense model aimed at local, private agent inference. Gemma 4 from Google DeepMind comes in two flavours: a 26B mixture-of-experts variant activating about 4B parameters per token, and a 31B dense variant. Qwen 3.6 27B from Alibaba holds the strong published coding evidence, with an official SWE-bench Verified score of 77.2.
All three are Apache 2.0, which is where the licence picture gets complicated. Muse Glimmer’s model card also links a separate Meta Usage Policy that layers acceptable-use conditions on top of the licence. It doesn’t remove commercial rights, but the licence isn’t the only document governing your deployment, which is part of the wider story of why Meta is giving the model away.
The workload mapping is clearer than the benchmark tables suggest. For local, private agent inference on a single 24GB card, Muse Glimmer is the fit, partly because it holds a 262,144-token context at the same VRAM budget where Qwen is limited to about 70,000. For multimodal and edge work, or MoE efficiency, Gemma 4 is the option. For coding-heavy work with hardware unconstrained, Qwen 3.6 is the incumbent.
Two durability caveats. Gemma 4’s 26B MoE variant activates about 4B parameters per token while the 31B dense sibling activates all of them, so don’t conflate the two. And Qwen’s rumoured 3.8 successor could shift the rankings quickly. Before you commit, evaluate trust, risk and TCO on your own hardware and scaffold, and run your own evals on the tasks that matter. That choice only matters if the model runs on the hardware you have.
How does a 30-billion-parameter model run on a single consumer GPU, and how does speculative decoding reach 20K tokens/sec? #
A 30B model runs on a single consumer GPU because 4-bit quantisation shrinks the weights to fit roughly 24GB of VRAM. It doesn’t fit by accident. At full precision the weights alone need over 55GB, so 4-bit quantisation is what makes local inference possible. Each weight shrinks from 16 bits to 4, dropping the weights to under 20GB and leaving headroom for the KV cache within a 24GB or 32GB envelope. The quality trade-off is modest: Meta reports minimal to no degradation on agentic tasks, and community measurements land around 1 to 3%.
The 20K tokens per second figure comes from speculative decoding, specifically Meta’s DFlash. A small drafter proposes blocks of tokens and the main model verifies them in parallel, so you accept multiple tokens per pass instead of one. Gemma 4 has a comparable technique in multi-token prediction, at 1.4 to 2.2x faster inference. Treat 20K as a best-case ceiling rather than an expected throughput. Measured DFlash results top out near 233 tokens per second on an RTX 5090, and in one measured configuration on a constrained 24GB Mac the drafter slowed generation down.
Your hardware matters as much as the model. A 24GB RTX 3090 or 4090 fits a 4-bit 30B model; Apple Silicon shares one unified-memory pool, so a 32GB+ Mac can too. The engine you choose changes what “runs locally” means: llama.cpp or Ollama suit single-user work, while vLLM and SGLang add continuous batching for team throughput. Local inference carries operational costs: model updates, security patches, monitoring and scaling. The capex versus cloud arithmetic is the other half of the same spreadsheet.
The three contenders map to different jobs, so the best pick is the one whose licence, VRAM footprint and tooling line up with the job you need done.
The mechanics are the point. 4-bit quantisation puts a 30B model in roughly 24GB of VRAM, and DFlash speculative decoding is what makes the 20K tokens per second claim credible. Feasibility depends on the engineering: quantisation, VRAM and decoding, rather than on vendor claims.
The deciding question is whether the operational load is worth the control. No per-token API cost and data that stays local are the upside, but they arrive with serving, update and security overhead. That’s the build-vs-buy decision behind the benchmark tables, and it lives inside Meta’s wider AI offensive.
Frequently Asked Questions #
Is Muse Glimmer truly open source, or is there a catch?
Apache 2.0 governs the weights, so commercial use, modification and redistribution are permitted without royalties. The catch is the Meta Usage Policy, which layers acceptable-use conditions on top of the licence rather than replacing it. For most CTOs the practical effect is small, but if your product sits in a restricted category, have counsel review both documents before you ship.
Will a 16GB GPU run a 30B model?
Not comfortably, and usually not at all. A 4-bit 30B model needs roughly 24GB of VRAM once the KV cache and activations are added, so an RTX 3090 or 4090 fits while a 16GB card forces off to system RAM and slow generation. Budget headroom for context length rather than matching the raw weight size exactly.
What are GGUF and Q4 formats, and why do they matter?
GGUF is the file format that packages quantised weights, and Q4 refers to 4-bit quantisation, the common setting for fitting a 30B model on consumer hardware. Together they determine what an engine can load and how much VRAM it needs. Choosing Q4 is a memory decision first, with a quality trade-off you accept to run locally.
What is the difference between a MoE and a dense model?
A dense model activates all of its parameters for every token, while a mixture-of-experts model activates only a subset. Gemma 4’s 26B MoE variant keeps roughly 4B parameters active per token, so it can run faster and cheaper than its size suggests. The trade-off is added complexity and more total memory, which matters when you are sizing a serving node.
Can I run Muse Glimmer on a Mac?
Yes, if the machine has enough unified memory. Apple Silicon shares one pool between CPU and GPU, so a Mac with 32GB or more can run a 4-bit 30B model where a discrete card with less VRAM cannot. Expect slower generation than a dedicated GPU, but it is a workable option for single-user testing and private workloads.
Is the 20K tokens per second claim realistic in the real world?
It is an aspirational ceiling, not a guaranteed number. The 20K tokens per second figure depends on hardware, engine, batch size and how often the drafter guesses correctly, and it reflects a best-case configuration. Most single-GPU setups will land well below it, so treat the figure as evidence of the technique’s headroom, not a purchasing benchmark.
Can I fine-tune these models, or are they inference-only?
All three are Apache 2.0 licensed weights, so fine-tuning is permitted. Muse Glimmer is positioned for local agent inference, but that does not block domain-specific fine-tuning. The real constraint is hardware: full fine-tuning of a 30B model needs far more memory than inference, so most teams reach for parameter-efficient methods such as LoRA instead.
What happens if I try to run a 30B model without quantisation?
You need roughly 60GB of memory just for the weights at full precision, plus the KV cache and activations, which is beyond a single consumer GPU. The model will not load cleanly; you will see out-of-memory errors or heavy off to system RAM and disk. Quantisation is not an optimisation here, it is what makes the model fit at all.
Are these models actually private when I run them locally?
Yes, with a caveat. When inference runs on your own hardware, prompts and outputs never leave the machine, which is the core data-residency benefit. The caveat is the surrounding stack: telemetry, model updates, monitoring tools and third-party engines can leak data unless you lock them down. Local inference is private by default, not private by accident.
How do I actually validate a vendor benchmark before committing?
Pick one hardware target, one inference engine and one scaffold, then run your own evals on the tasks that matter to your product. Vendor numbers are directional because they use different prompts, settings and datasets, so they rarely transfer cleanly. A small internal benchmark on your own data is the only way to turn a marketing claim into a purchasing decision.
What is the Meta Usage Policy, and does it limit commercial use?
The Meta Usage Policy sits alongside the Apache 2.0 licence and adds acceptable-use rules, for example around restricted categories, without removing commercial rights. It does not stop you from building a paid product on Muse Glimmer. It does mean the licence is not the only document governing your deployment, so include the policy in legal review.
What is the KV cache, and why does it affect my VRAM budget?
The KV cache stores the keys and values for tokens the model has already processed, so it does not recompute them for every new token. It grows with context length, which is why a 30B model’s memory use climbs as conversations get longer. A long context window needs VRAM headroom beyond the quantised weights, or generation slows and eventually fails.