How MiMo-V2.6 Grades Its Own Reasoning to Keep Improving

Xiaomi's MiMo-V2.6 models use groupwise agentic grading to replace binary pass/fail rewards in reinforcement learning, ranking multiple rollouts per task against each other to preserve gradient signal after models pass 80-90% of rollouts. The method combines Groupwise Reward Synthesis (GRS), which builds task-specific rubrics offline from contrasting rollouts, and Groupwise Advantage Redistribution (GAR), which ranks passing trajectories online during training. Both checkpoints, MiMo-V2.6-Pro-RL (1.02T total / 42B activated parameters, Mixture of Experts) and MiMo-V2.6-Flash-RL (309B total / 15B activated parameters), share the same training recipe, with a single mixed RL run training coding, general agent, visual, and cybersecurity tasks on batches of 1,568 prompts times 16 rollouts per step, followed by MOPD2 (Multi-Prefix Multi-Teacher On-Policy Distillation).

How MiMo-V2.6 Grades Its Own Reasoning to Keep Improving Xiaomi's MiMo-V2.6 uses groupwise agentic grading and distillation to scale RL past binary rewards. Here's how the method works. What is groupwise agentic grading in MiMo-V2.6? Groupwise agentic grading is the reward-scoring method behind Xiaomi’s MiMo-V2.6 models. Instead of marking a rollout simply pass or fail, an agentic grader compares multiple rollouts generated for the same task and ranks them against each other. This produces a reward signal with more resolution than binary pass/fail, which lets the reinforcement learning RL loop distinguish a clever, efficient solution from a clumsy one that happened to also pass. Xiaomi built this into both public checkpoints, MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL, as part of a broader effort to scale RL training toward what the company calls self-improvement. TL;DR - Binary rewards cap RL scaling because once a model passes most tasks in a domain, pass/fail signals stop distinguishing good solutions from mediocre ones, so Xiaomi replaced that signal with graded comparison inside each rollout group. - Groupwise Reward Synthesis GRS builds task-specific rubrics offline from contrasting rollouts, then blends rubric-based quality scores with hard test outcomes for a richer reward. - Groupwise Advantage Redistribution GAR works online during training, ranking passing trajectories within a batch and shifting more advantage toward the higher-quality ones instead of splitting credit evenly. - The grader judges the policy against its own outputs , which closes a self-improvement loop where the model’s better attempts become the training signal that shapes its worse ones. - A single mixed RL run , nicknamed “You Only RL Once,” trains coding, general agent, visual, and cybersecurity tasks together in the same batches so skills transfer across domains and harnesses the model never saw during training. - MOPD2 Multi-Prefix Multi-Teacher On-Policy Distillation runs after the main RL phase to extend gains into tasks that are hard to verify automatically, using teacher and SFT demonstration prefixes to train decision points without regenerating full trajectories. - Both checkpoints share the exact same training recipe : Pro 1.02T total / 42B activated parameters, Mixture of Experts and Flash 309B total / 15B activated parameters differ in size and speed, not in method. - ✕a coding agent - ✕no-code - ✕vibe coding - ✕a faster Cursor The one that tells the coding agents what to build. Why doesn’t binary pass/fail work for scaling RL? Most RL training for coding and agentic tasks uses a simple correctness check: did the code pass the test suite, did the agent complete the task, yes or no. That works early on, when models fail most attempts and pass/fail gives useful gradient signal. The problem shows up as models get better. Once a model solves 80 or 90 percent of a training task’s rollouts, the reward signal collapses. Every passing rollout gets the same score, whether it took the shortest path or a bloated, inefficient one, whether it used ten tool calls or a hundred. The RL optimizer has nothing left to push toward. MiMo-V2.6 addresses this by treating the group of rollouts for a single task as the unit of comparison rather than scoring each rollout in isolation. Given a batch of GRPO rollouts Group Relative Policy Optimization, the mixed multi-domain RL trained on batches of 1,568 prompts times 16 rollouts per step , an agentic grader looks across the whole group and ranks the solutions relative to each other. That relative signal survives even after most rollouts pass, because “better” is now a spectrum instead of a binary flag. How does Groupwise Reward Synthesis work? Groupwise Reward Synthesis GRS operates offline, before the reward is applied during training. For a given task, the system collects a set of contrasting rollouts, meaning solutions that differ meaningfully in approach or quality, and uses them to build a task-specific rubric. Rather than a generic rubric written by a human in advance, the rubric is synthesized from what actually separates good and bad solutions in that specific task’s rollout pool. That rubric-based quality judgment is then fused with the hard test outcome did it actually pass or fail . The result is a reward that respects ground truth correctness but adds gradation on top of it, so two passing rollouts can still receive different rewards based on how well they satisfy the rubric. This matters most in domains like general agent tasks or coding, where “correct” doesn’t mean “equally good,” and where correctness alone stops giving the optimizer anything to chase once pass rates climb. How does Groupwise Advantage Redistribution differ from GRS? Where GRS works offline to build the rubric, Groupwise Advantage Redistribution GAR works online, during the RL update step itself. GAR takes the passing trajectories within a batch and ranks them, then redistributes the credit the “advantage” in policy gradient terms toward the higher-quality trajectories rather than spreading it evenly across everything that passed. Remy doesn't write the code. It manages the agents who do. Remy runs the project. The specialists do the work. You work with the PM, not the implementers. In practice this means the optimizer gets pushed harder toward the rollouts the grader judged best, and pulled less hard, or not at all, toward rollouts that merely passed. According to Xiaomi’s own description, this steering effect shows up concretely: the model gets pushed toward shorter reasoning paths and fewer tokens spent per task, because efficient correct solutions outrank correct-but-wasteful ones in the ranking. That’s a direct answer to a common complaint about RL-trained reasoning models, which is that they tend to ramble or take needlessly long paths to a correct answer. Together, GRS and GAR form what Xiaomi calls the self-improvement loop: the model’s own rollouts, judged against each other by an agentic grader, become the mechanism that sharpens the model’s future rollouts. There’s no external gold-standard dataset providing the comparison; the policy is judged against its own distribution of outputs. What keeps this self-referential loop from gaming its own grader? A model being judged by a grader trained on its own outputs is a setup ripe for reward hacking, where the policy learns to satisfy the grader’s proxy signal without actually getting better at the underlying task. Xiaomi’s documentation describes several guardrails built into the training pipeline to keep this from happening: environment hardening, adversarial screening, and verifier cross-checks running throughout the RL process specifically to catch and correct exploits of the reward signal. There’s also a distinct alignment step called “Aligned RL,” which uses a cold start built from self-correction: the model reflects on and rewrites its own misaligned turns into more grounded next steps before the main RL phase begins. This is separate from the groupwise grading mechanism but serves a related purpose, keeping the training loop pointed at genuinely useful behavior rather than whatever maximizes the grader’s score. What is “You Only RL Once” and why does it matter? Rather than running separate RL passes for coding, general agent tasks, visual tasks, and cybersecurity, MiMo-V2.6 trains all four in a single mixed run, with different task types and multiple agent harnesses mixed into the same training batches. Xiaomi’s stated rationale is that capabilities reinforce each other across domains, and strategies learned in one harness transfer to harnesses the model never encountered during training. This is a meaningful design choice distinct from the grading mechanism itself. Groupwise agentic grading gives the model a sharper signal within any given task; mixing domains in one run is what lets skill in, say, tool-calling for a coding agent carry over to a completely different agentic environment. The two together are what Xiaomi frames as “scaling RL compute, environment diversity, and grader compute together.” What is MOPD2 and what problem does it solve? Multi-Prefix Multi-Teacher On-Policy Distillation MOPD2 is a separate training stage that runs after the main mixed RL phase. Groupwise grading works well when there’s a way to compare rollouts, but some tasks are hard to verify automatically at all, so there’s little for a grader agentic or otherwise to work with. MOPD2 addresses that gap by combining the student model’s own autonomous rollouts with prefix-conditioned single-turn rollouts drawn from two sources: Teacher-Prefix rollouts reusing history from stronger teacher model trajectories and SFT-Prefix rollouts reusing history from supervised fine-tuning demonstrations . By reusing prior context rather than regenerating it, the model can train specifically on the decision points that matter without having to replay entire trajectories from scratch. The stated goal is extending the capability gains from RL into tasks that resist automatic verification, which is where pure reward-based RL, however finely graded, has less to work with. Frequently Asked Questions What’s the difference between MiMo-V2.6-Pro-RL and MiMo-V2.6-Flash-RL? Both use the identical RL training recipe described here, including groupwise agentic grading and MOPD2. They differ in scale: Pro is a Mixture-of-Experts model with 1.02 trillion total parameters and 42 billion activated per token, while Flash has 309 billion total and 15 billion activated, making Flash the faster, lighter-weight option. Is groupwise agentic grading the same as using an LLM-as-judge? It’s related but more structured. Rather than asking a judge model to score a single rollout in isolation, the grader builds rubrics from contrasting rollouts within the same group GRS and ranks passing trajectories relative to each other during training GAR , producing a comparative signal rather than an absolute one. Does this method require human-written rubrics for every task? No. GRS synthesizes rubrics from contrasting rollouts within the task’s own rollout pool rather than relying on rubrics written in advance by humans, which is part of what lets it scale across many task domains at once. Why does the model end up more token-efficient after this training? Because GAR redistributes advantage toward higher-ranked trajectories among those that pass, and shorter, more direct solutions tend to rank above correct-but-meandering ones. The optimizer is pushed toward the paths the grader consistently ranks better, which in practice favors brevity. Can this method be reused for domains outside coding and agents? The architecture is domain-general in principle, since groupwise comparison doesn’t depend on any single task type. Xiaomi applied it across coding, general agent tasks, visual tasks, and cybersecurity within a single mixed RL run, suggesting the approach is meant to generalize rather than being hand-tuned to one domain.