{"slug": "mimo-v2-6-pro-architecture-and-training-notes", "title": "MiMo-V2.6 Pro Architecture and Training Notes", "summary": "Xiaomi's MiMo-V2.6 Pro ranks No. 1 on open-weight benchmarks by weighted average despite using a classic Grouped Query Attention architecture with Sliding Window Attention at a 128-token window, according to training notes published by Sebastian Raschka. Xiaomi's MiMo team reported in its technical report that DeepSWE pass@1 accuracy on held-out harnesses improved from approximately 50% to 66%, with training using 1,568 prompts × 16 rollouts = 25,088 trajectories and 2.7–3.7 billion training tokens per update. The team also replaced a simple correctness verifier with an agentic grader that examines execution traces, which Raschka cites as evidence that data and post-training recipe improvements, not attention variants, drive most progress.", "body_md": "# MiMo-V2.6 Pro Architecture and Training Notes\n\nXiaomi’s new MiMo-V2.6 Pro is “simply” the best (for now). Despite its simple architecture design it’s currently No.1 in the open-weight benchmarks (weighted average).\n\nWith “simple,” I mean a classic [Grouped Query Attention (GQA)](https://sebastianraschka.com/llm-architecture-gallery/gqa/) with [Sliding Window Attention (SWA)](https://sebastianraschka.com/llm-architecture-gallery/swa/) at a tiny 128-token window size.\n\nSo, that underlines one of the points I’ve been trying to make in recent months: most of the progress still comes from the data and post-training recipe improvements. Fancy [attention variants](https://magazine.sebastianraschka.com/p/visual-attention-variants) are just mostly efficiency tweaks.\n\nWhat are some of the training data improvements and recipe improvements? The MiMo team shared a pretty detailed [technical report](https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL/blob/main/MiMo_V2_6_technical_report.pdf). Lots to carefully digest there, but in short, there are a few things that stood out:\n\n1. \nAn increase in agent tasks; also training across different harnesses (the average DeepSWE pass@1 accuracy on held-out harnesses improved from approximately 50% -> 66%).\n2. \nBetter reward signals: they replaced a simple correctness verifier with an agentic grader that looks at the execution traces as well.\n3. \nLarge [RL](https://magazine.sebastianraschka.com/p/the-state-of-llm-reasoning-model-training) batches (1,568 prompts × 16 rollouts = 25,088 trajectories) and 2.7–3.7 billion training tokens per update (unclear, though, what the predecessor used).\n\nSource: website version of my [Substack note](https://substack.com/@rasbt/note/c-343108099).\n\n## Read Next\n\n[It's Easy to Dismiss Jev as Just a Classifier A short note on Jev's generalization, possible encoder-style architecture and training, and Choice and Noul API examples.](https://sebastianraschka.com/blog/2026/jev-classification-generalization.html)\n\n[Pacing != Pacing Development My take on AI model pacing as a framework for release checks and the competitive pressure around model releases.](https://sebastianraschka.com/blog/2026/pacing-development.html)\n\n[AI Reasoning Models Course on LinkedIn Learning A 90-minute LinkedIn Learning course on how reasoning models relate to conventional LLMs and how they are developed.](https://sebastianraschka.com/blog/2026/ai-reasoning-models-course.html)", "url": "https://wpnews.pro/news/mimo-v2-6-pro-architecture-and-training-notes", "canonical_source": "https://sebastianraschka.com/blog/2026/mimo-v2-6-pro-architecture-training-notes.html", "published_at": "2026-09-22 13:47:58+00:00", "updated_at": "2026-09-22 14:24:19.514273+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "machine-learning", "ai-agents"], "entities": ["Xiaomi", "MiMo-V2.6 Pro", "MiMo-V2.6-Flash-RL", "Sebastian Raschka", "Grouped Query Attention", "Sliding Window Attention", "DeepSWE", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/mimo-v2-6-pro-architecture-and-training-notes", "markdown": "https://wpnews.pro/news/mimo-v2-6-pro-architecture-and-training-notes.md", "text": "https://wpnews.pro/news/mimo-v2-6-pro-architecture-and-training-notes.txt", "jsonld": "https://wpnews.pro/news/mimo-v2-6-pro-architecture-and-training-notes.jsonld"}}