MiMo-V2.6 Pro Architecture and Training Notes

Xiaomi's MiMo-V2.6 Pro ranks No. 1 on open-weight benchmarks by weighted average despite using a classic Grouped Query Attention architecture with Sliding Window Attention at a 128-token window, according to training notes published by Sebastian Raschka. Xiaomi's MiMo team reported in its technical report that DeepSWE pass@1 accuracy on held-out harnesses improved from approximately 50% to 66%, with training using 1,568 prompts × 16 rollouts = 25,088 trajectories and 2.7–3.7 billion training tokens per update. The team also replaced a simple correctness verifier with an agentic grader that examines execution traces, which Raschka cites as evidence that data and post-training recipe improvements, not attention variants, drive most progress.

MiMo-V2.6 Pro Architecture and Training Notes Xiaomi’s new MiMo-V2.6 Pro is “simply” the best for now . Despite its simple architecture design it’s currently No.1 in the open-weight benchmarks weighted average . With “simple,” I mean a classic Grouped Query Attention GQA https://sebastianraschka.com/llm-architecture-gallery/gqa/ with Sliding Window Attention SWA https://sebastianraschka.com/llm-architecture-gallery/swa/ at a tiny 128-token window size. So, that underlines one of the points I’ve been trying to make in recent months: most of the progress still comes from the data and post-training recipe improvements. Fancy attention variants https://magazine.sebastianraschka.com/p/visual-attention-variants are just mostly efficiency tweaks. What are some of the training data improvements and recipe improvements? The MiMo team shared a pretty detailed technical report https://huggingface.co/XiaomiMiMo/MiMo-V2.6-Flash-RL/blob/main/MiMo V2 6 technical report.pdf . Lots to carefully digest there, but in short, there are a few things that stood out: 1. An increase in agent tasks; also training across different harnesses the average DeepSWE pass@1 accuracy on held-out harnesses improved from approximately 50% - 66% . 2. Better reward signals: they replaced a simple correctness verifier with an agentic grader that looks at the execution traces as well. 3. Large RL https://magazine.sebastianraschka.com/p/the-state-of-llm-reasoning-model-training batches 1,568 prompts × 16 rollouts = 25,088 trajectories and 2.7–3.7 billion training tokens per update unclear, though, what the predecessor used . Source: website version of my Substack note https://substack.com/@rasbt/note/c-343108099 . Read Next It's Easy to Dismiss Jev as Just a Classifier A short note on Jev's generalization, possible encoder-style architecture and training, and Choice and Noul API examples. https://sebastianraschka.com/blog/2026/jev-classification-generalization.html Pacing = Pacing Development My take on AI model pacing as a framework for release checks and the competitive pressure around model releases. https://sebastianraschka.com/blog/2026/pacing-development.html AI Reasoning Models Course on LinkedIn Learning A 90-minute LinkedIn Learning course on how reasoning models relate to conventional LLMs and how they are developed. https://sebastianraschka.com/blog/2026/ai-reasoning-models-course.html