{"slug": "model-mimo-v2-6-distill-qwen-9b", "title": "Model: MiMo-V2.6-Distill-Qwen-9B", "summary": "Xiaomi released MiMo-V2.6-Distill-Qwen-9B in September, a 9B-parameter distillation of its MiMo-V2.6 model into the Qwen3.5 architecture with a 262K maximum context and image input through a separate vision encoder, according to the model listing. Only 8 of the model's 32 layers keep a cache that grows with context, and bartowski's GGUF builds put Q4_K_M at about 6.3 GB of VRAM at 4K context and 6.7 GB at 16K, comfortable on an 8 GB card, while Q8_0 needs about 10.1 GB at 4K and the vision projector adds about 0.92 GB as a separate download. No per-level quality loss has been published for the four quantization variants (Q3_K_M, Q4_K_M, Q5_K_M, Q8_0).", "body_md": "# MiMo-V2.6-Distill-Qwen-9B\n\n9B\nXiaomi MiMo\n\nXiaomi's MiMo-V2.6 distilled into the Qwen3.5 9B architecture, released in September, with image input through a separate vision encoder. Only 8 of its 32 layers keep a cache that grows with context. At Q4_K_M it needs about 6.3 GB at 4K and 6.7 GB at 16K, comfortable on an 8 GB card; at 32K (7.2 GB) it is tight on an 8 GB card and comfortable from 10 GB up. It needs about 10.5 GB at 128K, comfortable on a 12 GB card. Q8_0 needs about 10.1 GB at 4K, comfortable on a 12 GB card. The vision encoder adds about 0.9 GB. Sizes come from bartowski's GGUF files, and no per-level quality loss has been published.\n\n262K\n\nMax Context\n\n4\n\nQuant Variants\n\nGGUF Q8_0\n\nBest Quality\n\n—\n\nPerplexity rise (best build)\n\n## Quantization Variants\n\nPer-quant VRAM, quality loss, and inference speed on RTX 4090\n\nMeasured = run on this site · Estimated = not run here: calculated, or a figure this site has not verified · Community = public reports\n\n| Format | Level | BPW | VRAM | PPL Loss | Speed | Source | Actions | \n|---|---|---|---|---|---|---|---|\n| GGUF | Q3_K_M | 4 | 4.8 GB | — | — | Estimated |  | \n| GGUF | Q4_K_M | 5.22 | 6.3 GB | — | — | Estimated |  | \n| GGUF | Q5_K_M | 6.14 | 7.3 GB | — | — | Estimated |  | \n| GGUF | Q8_0 | 8.52 | 10.1 GB | — | — | Estimated |  | \n\n### Which file each size is for\n\nA file size is not a memory figure: the VRAM column is the calculator's total at 4K context — weights plus KV cache plus a 10% activation buffer. Units differ on purpose: file sizes are decimal GB, as Hugging Face lists them; the VRAM column is GiB (1024³ bytes) with 2% added to the weights, so 7.12 GB of file counts as about 6.8 GiB there. A vision projector (mmproj), where the repo ships one, is a separate download and is in neither number.\n\n- GGUF Q3_K_M[bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF](https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF) · MiMo-V2.6-Distill-Qwen-9B-Q3_K_M.gguf · 4.48 GB (4,479,948,320 bytes) · + vision projector 0.92 GB, separate · listing read 2026-10-29\n- GGUF Q4_K_M[bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF](https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF) · MiMo-V2.6-Distill-Qwen-9B-Q4_K_M.gguf · 5.84 GB (5,841,049,120 bytes) · + vision projector 0.92 GB, separate · listing read 2026-10-29\n- GGUF Q5_K_M[bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF](https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF) · MiMo-V2.6-Distill-Qwen-9B-Q5_K_M.gguf · 6.88 GB (6,876,124,704 bytes) · + vision projector 0.92 GB, separate · listing read 2026-10-29\n- GGUF Q8_0[bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF](https://huggingface.co/bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF) · MiMo-V2.6-Distill-Qwen-9B-Q8_0.gguf · 9.55 GB (9,545,979,424 bytes) · + vision projector 0.92 GB, separate · listing read 2026-10-29\n\n## Similar models\n\n[Compare with MiMo-V2.6-Flash](https://quantized.uk/tools/compare/?a=mimo-v2-6-distill-qwen-9b&b=mimo-v2-6-flash)\n\n[309B-A15BPro GPUMac / Apple SiliconMXFP4175.3 GB+0.0%perplexity rise](https://quantized.uk/quant-hub/mimo-v2-6-flash/)\n\n### MiMo-V2.6-Flash\n\nXiaomi MiMo\n\nXiaomi's smaller MiMo-V2.6 model: 309B parameters with about 15B active per token, MIT licence. It reads text, images, video and audio. The released weights are already 4-bit on the experts (MXFP4), and ggml-org's GGUF keeps them that way: about 175 GB at 4K context. That is comfortable only on a 512 GB Mac Studio, and tight on a 256 GB one. The Q2_K build (about 132 GB) fits a 256 GB Mac comfortably and is tight on a 192 GB one. Only 9 of its 48 layers keep a growing cache, so even the full 1M window adds about 25 GB. llama.cpp runs it as `mimo2`, the same as MiMo-V2.6-Pro, and vision loads through a separate encoder. Sizes come from ggml-org's GGUF files; no per-level quality loss has been published.\n\n[1T-A42BPro GPUMXFP4603.8 GB+0.0%perplexity rise](https://quantized.uk/quant-hub/mimo-v2-6-pro/)\n\n### MiMo-V2.6-Pro\n\nXiaomi MiMo\n\nXiaomi's open-weight flagship, near the top of the open-model rankings this autumn. It has about 1.02 trillion parameters, roughly 42B of them active per token, and ships under the MIT licence. The released weights are already 4-bit (MXFP4 experts), and the GGUF builds repack them rather than quantize again. At that level it needs about 604 GB at 4K context, more than any single machine in this index. The largest Mac Studio gives its GPU about 384 GB, and four 80 GB H100s hold 320 GB. Eight 80 GB cards (640 GB) are about the smallest server that holds it, and even that is tight on this site's rule. It is listed so the calculator can say so. Only 10 of its 70 layers keep a growing cache, so even 1M tokens adds only about 50 GB. llama.cpp runs it as `mimo2`, and vision loads through a separate encoder. Sizes come from a published GGUF file; no per-level quality loss has been published.\n\n[8BConsumer GPUMac / Apple SiliconQ4_K_M6.2 GB+3.0%perplexity rise](https://quantized.uk/quant-hub/qwen3-vl-8b/)\n\n### Qwen3-VL 8B Instruct\n\nAlibaba Qwen3-VL\n\nCurrent-generation vision-language model that still fits a single 8–12GB card at Q4 (~5.9GB). The realistic multimodal option for people without a 24GB GPU — note the vision encoder adds VRAM the KV-cache math below does not model.\n\n[9BConsumer GPUMac / Apple SiliconQ4_K_M6.6 GB—perplexity rise](https://quantized.uk/quant-hub/qwen3-5-9b/)\n\n### Qwen3.5 9B\n\nAlibaba Qwen3.5\n\nThe newest small Qwen and a common pick for 8 to 16 GB cards. Only 8 of its 32 layers keep a growing KV cache, so it can use a long context on modest hardware. At Q4_K_M it needs about 6.6 GB at 4K and 7.0 GB at 16K, comfortable on an 8 GB card; at 32K (7.5 GB) it is tight there and comfortable from 10 GB up. Its full 256K window needs about 15.2 GB. Q8_0 needs about 10.4 GB, comfortable from 12 GB up. Reasoning is off by default on the small Qwen3.5 sizes, and it reads images through a separate vision encoder. Sizes come from the published GGUF file sizes, and no per-level quality loss has been published.\n\n## Running MiMo-V2.6-Distill-Qwen-9B locally\n\nAt Q4_K_M and 4K of context, MiMo-V2.6-Distill-Qwen-9B needs about 6.3 GB — 5.5 GB of weights, 0.13 GB of KV cache and a 0.6 GB activation buffer. The smallest cards in this index that clear that comfortably are 8 GB ones — 9 of them, from the RTX 4060 up — and 76 of the 78 cards here do. These are calculated figures, not measurements: the estimate stops counting a card as comfortable at 88% of its VRAM, which is roughly the room a desktop session needs.\n\n## What longer context costs\n\nGoing from 4K to 32K adds about 0.87 GB, taking the total to 7.2 GB. That is gentler than the parameter count suggests: only 8 of 32 layers keep a KV cache that grows with context, so the cache scales at a fraction of the usual rate. The model's native window is 256K; holding all of it at Q4_K_M would need about 15 GB.\n\n## Which build to download\n\nThis index only tracks MiMo-V2.6-Distill-Qwen-9B in GGUF, across 4 levels (Q3_K_M, Q4_K_M, Q5_K_M, Q8_0). No per-level perplexity sweep has been published for this model, so the quality column is empty rather than estimated — within one model, more bits per weight is the only ordering the data supports. No throughput has been measured for this model on this site's hardware. GGUF runs on llama.cpp and Ollama across NVIDIA, AMD and Apple silicon; AWQ and GPTQ target vLLM on CUDA and ROCm; EXL2 is ExLlamaV2 — CUDA only, and now archived, so local use rather than serving.\n\n## Common questions\n\n- How much VRAM does MiMo-V2.6-Distill-Qwen-9B need?\n- About 6.3 GB at Q4_K_M with 4K of context and batch 1, rising to roughly 7.2 GB at 32K. That figure is weights plus KV cache plus a 10% activation buffer, calculated from the model's architecture rather than measured on a card.\n- Will MiMo-V2.6-Distill-Qwen-9B run on a 8GB GPU?\n- Yes — at Q4_K_M and 4K context it needs about 6.3 GB, which leaves 1.8 GB spare on a RTX 4060, or on any of the other 8 GB cards here (9 in all). That is the smallest size in this index that clears it comfortably; 76 of 78 do.\n- Which quantization of MiMo-V2.6-Distill-Qwen-9B should I use?\n- No published quality comparison exists for this model, so pick by footprint: Q4_K_M is the level most people run, and a higher bits-per-weight level is more faithful. All 4 levels in the index are Q3_K_M, Q4_K_M, Q5_K_M, Q8_0.\n\n## Where this model fits\n\nSized at Q4_K_M with a 4K context window, smallest card first. Comfortable means the estimate uses at most 88% of the memory.\n\n- Runs comfortably on\n- [RTX 40608GB · +1.8](https://quantized.uk/gpu/rtx-4060/)[RTX 4060 Ti 8G8GB · +1.8](https://quantized.uk/gpu/rtx-4060-ti-8g/)[Radeon RX 9060 XT 8G8GB · +1.8](https://quantized.uk/gpu/radeon-rx-9060-xt-8g/)[RTX 5060 Ti 8G8GB · +1.8](https://quantized.uk/gpu/rtx-5060-ti-8g/)[RTX 50608GB · +1.8](https://quantized.uk/gpu/rtx-5060/)\n- Tight but possible\n- [Mac M1 8G8GB](https://quantized.uk/gpu/mac-m1-8g/)[Mac M3 8G8GB](https://quantized.uk/gpu/mac-m3-8g/)\n- Ships in\n- [GGUF](https://quantized.uk/formats/gguf/)\n- Try it yourself\n- [Size it in the calculator](https://quantized.uk/tools/vram-calc/?model=mimo-v2-6-distill-qwen-9b&quant=Q4_K_M)[Generate the run command](https://quantized.uk/tools/cli-gen/?model=mimo-v2-6-distill-qwen-9b&quant=Q4_K_M)\n\n## How to actually run this\n\nDeployment guides for this model and this class of hardware.\n\nThis page's figures change when the model or the runtime does.Last updated 2026-10-29 [RSS → /feed.xml](https://quantized.uk/feed.xml)", "url": "https://wpnews.pro/news/model-mimo-v2-6-distill-qwen-9b", "canonical_source": "https://quantized.uk/quant-hub/mimo-v2-6-distill-qwen-9b/", "published_at": "2026-10-29 03:45:40.510494+00:00", "updated_at": "2026-10-29 03:45:42.623165+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-tools"], "entities": ["Xiaomi", "MiMo-V2.6-Distill-Qwen-9B", "Qwen3.5", "bartowski", "Hugging Face", "MiMo-V2.6-Flash", "MiMo-V2.6-Pro"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/model-mimo-v2-6-distill-qwen-9b", "markdown": "https://wpnews.pro/news/model-mimo-v2-6-distill-qwen-9b.md", "text": "https://wpnews.pro/news/model-mimo-v2-6-distill-qwen-9b.txt", "jsonld": "https://wpnews.pro/news/model-mimo-v2-6-distill-qwen-9b.jsonld"}}