309B-A15B Xiaomi MiMo
Xiaomi's smaller MiMo-V2.6 model: 309B parameters with about 15B active per token, MIT licence. It reads text, images, video and audio. The released weights are already 4-bit on the experts (MXFP4), and ggml-org's GGUF keeps them that way: about 175 GB at 4K context. That is comfortable only on a 512 GB Mac Studio, and tight on a 256 GB one. The Q2_K build (about 132 GB) fits a 256 GB Mac comfortably and is tight on a 192 GB one. Only 9 of its 48 layers keep a growing cache, so even the full 1M window adds about 25 GB. llama.cpp runs it as mimo2, the same as MiMo-V2.6-Pro, and vision loads through a separate encoder. Sizes come from ggml-org's GGUF files; no per-level quality loss has been published.
1049K
Max Context
2
Quant Variants
GGUF MXFP4
Best Quality
100.0%
Accuracy Retained
Quantization Variants #
Per-quant VRAM, quality loss, and inference speed on RTX 4090
Measured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports
| Format | Level | BPW | VRAM | PPL Loss | Speed | Source | Actions |
|---|---|---|---|---|---|---|---|
| GGUF | Q2_K | 3.27 | 132.1 GB | — | — | Estimated | |
| GGUF | MXFP4 | 4.34 | 175.3 GB | 0.0% | — | Estimated |
Which file each size is for
A file size is not a memory figure: the VRAM column adds the KV cache at 4K context and runtime buffers to the weights. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.
- GGUF Q2_Kggml-org/MiMo-V2.6-Flash-RL-GGUF · MiMo-V2.6-Flash-RL-Q2_K-00001-of-00002.gguf · 126.37 GB (126,365,949,248 bytes) · 2 parts · + vision projector 2.75 GB, separate · listing read 2026-10-20
- GGUF MXFP4ggml-org/MiMo-V2.6-Flash-RL-GGUF · MiMo-V2.6-Flash-RL-MXFP4-00001-of-00002.gguf · 167.37 GB (167,369,465,152 bytes) · 2 parts · + vision projector 2.75 GB, separate · listing read 2026-10-20
Similar models #
[Compare with MiMo-V2.6-Pro](https://quantized.uk/tools/compare/?a=mimo-v2-6-flash&b=mimo-v2-6-pro)
[1T-A42BPro GPU603.8 GBmin VRAM100.0%accuracy](https://quantized.uk/quant-hub/mimo-v2-6-pro/)
### MiMo-V2.6-Pro
Xiaomi MiMo
Xiaomi's open-weight flagship, near the top of the open-model rankings this autumn. It has about 1.02 trillion parameters, roughly 42B of them active per token, and ships under the MIT licence. The released weights are already 4-bit (MXFP4 experts), and the GGUF builds repack them rather than quantize again. At that level it needs about 604 GB at 4K context, more than any single machine in this index. The largest Mac Studio gives its GPU about 384 GB, and four 80 GB H100s hold 320 GB. Eight 80 GB cards (640 GB) are about the smallest server that holds it, and even that is tight on this site's rule. It is listed so the calculator can say so. Only 10 of its 70 layers keep a growing cache, so even 1M tokens adds only about 50 GB. llama.cpp runs it as mimo2, and vision loads through a separate encoder. Sizes come from a published GGUF file; no per-level quality loss has been published.
816B-A42BMac / Apple SiliconPro GPU318.9 GBmin VRAM—accuracy
Muse Spark Open 816B-A42B
Meta Muse
Meta's largest open-weight model, released in October under the Muse Model License. It is an 816B mixture-of-experts model with about 42B parameters active per token, built for coding and agent work, with image and audio input. It is a single-machine model only at its smallest build. At Q2_K (ggml-org's 304 GB file) it needs about 319 GB at 4K context and still only 327 GB at 128K, because just 16 of its 62 layers keep a growing cache — comfortable on a 512 GB Mac Studio and tight across four 80 GB H100s. The full 1M window brings it to about 389 GB. Q4_K_S (470 GB) and Q8_0 (872 GB) fit no single machine in this index. The vision and audio encoders ship as separate files of 2–4 GB and are not included. Sizes come from the published GGUF files; Meta's benchmark claims are its own and no per-level quality loss has been published. The licence is not a standard open-source one: Meta's repository is gated, so downloads need an accepted access request; products built on it must show "Built with Muse Spark Open"; and a company with over $20M in annual revenue must sign a separate agreement with Meta before offering it as an API service.
74B-A9BConsumer GPUMac / Apple Silicon23.1 GBmin VRAM—accuracy
Nemotron 3.5 Bolt 74B-A9B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Bolt is a 74B mixture-of-experts model with about 9B parameters active per token, built to fit a 24 GB card. It is compressed from Nemotron 3.5 Super VL and ships in one build, NVIDIA's own native 2-bit file of 22.0 GB. Only 8 of its 88 layers keep a growing cache, so context costs little: about 23.1 GB at 4K, 24.1 GB at 128K and 25.2 GB at its full 256K window. That is the 24 GB NVIDIA designed for, but over this site's comfortable line, so a 24 GB card reads as tight; a 32 GB card or a 48 GB Mac runs it comfortably. The image encoder is a separate 1.7 GB file. NVIDIA's benchmark claims are its own, and no quality loss for the 2-bit build has been published.
120B-A12BMac / Apple SiliconPro GPU75.1 GBmin VRAM—accuracy
Nemotron 3.5 Super VL 120B-A12B
NVIDIA Nemotron
NVIDIA's Nemotron 3.5 Super VL is a 120B mixture-of-experts vision-language model with about 12B parameters active per token. It is built for coding, tool use and document work, with a context window of up to 1M tokens. Only 8 of its 88 layers keep a growing cache, so even the full window is cheap: at Q4_0 (ggml-org's 71.8 GB file) it needs about 75 GB at 4K and 84 GB at 1M. That runs comfortably on a 128 GB Mac or a DGX Spark 128G, and tightly on one 80 GB card. Q8_0 (128 GB) wants a 256 GB Mac. The vision encoder is a separate 1–1.7 GB file. Sizes come from the published GGUF files; NVIDIA's benchmark claims are its own and no per-level quality loss has been published.
Running MiMo-V2.6-Flash locally #
At MXFP4 and 4K of context, MiMo-V2.6-Flash needs about 175.3 GB — 159.1 GB of weights, 0.23 GB of KV cache and a 15.9 GB activation buffer. MXFP4 is the format MiMo-V2.6-Flash was released in, so it is the reference here rather than a community requantization. The smallest card in this index that clears that comfortably is the Mac M5 Ultra 512G at 512 GB, and 1 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.
What longer context costs #
Going from 4K to 32K adds about 0.62 GB, taking the total to 176.0 GB. That is gentler than the parameter count suggests: only 9 of 48 layers keep a KV cache that grows with context, and 39 more cap theirs at a sliding window, so the cache scales at a fraction of the usual rate. The model's native window is 1M; holding all of it at MXFP4 would need about 200 GB.
Which build to download #
This index only tracks MiMo-V2.6-Flash in GGUF, across 2 levels (Q2_K, MXFP4). MXFP4 carries the lowest published perplexity loss at 0.0%. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.
Common questions #
- How much VRAM does MiMo-V2.6-Flash need?
- About 175.3 GB at MXFP4 with 4K of context and batch 1, rising to roughly 176.0 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.
- Will MiMo-V2.6-Flash run on a 512GB GPU?
- Yes — at MXFP4 and 4K context it needs about 175.3 GB, which leaves 208.7 GB spare on a Mac M5 Ultra 512G. That is the smallest card in this index that clears it comfortably; 1 of 78 do.
- Which quantization of MiMo-V2.6-Flash should I use?
- MXFP4 has the lowest published quality loss (0.0%), and MXFP4 is the level most people run. All 2 levels in the index are Q2_K, MXFP4. Note this is a mixture-of-experts model — all parameters must be resident even though only a fraction are active per token, so the memory cost follows the total, not the active count.
Where this model fits #
Sized at MXFP4 with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.
-
Runs comfortably on
-
Tight but possible
-
Ships in
-
Try it yourself
How to actually run this #
Deployment guides for this model and this class of hardware.
This page's figures change when the model or the runtime does.Last updated 2026-10-23 RSS → /feed.xml