{"slug": "extremely-sparse-supervision-incentivizes-reasoning-ability", "title": "Extremely Sparse Supervision Incentivizes Reasoning Ability", "summary": "A new arXiv paper (2609.04565v1) reports that large language models can be effectively trained for reasoning with as few as one or two tokens per trajectory, just 0.05% of all generated tokens, challenging the assumption that post-training must be token-intensive. The finding, observed across nine teacher–student configurations using the Qwen3 family on mathematical reasoning and validated on coding, Llama models, and PPO-based RLVR, suggests that extremely sparse supervision can match or surpass full-token training in improving reasoning ability.", "body_md": "arXiv:2609.04565v1 Announce Type: new \nAbstract: Large language models demonstrate increasingly strong reasoning capabilities through effective post-training. Yet, prevailing post-training methods optimize over massive numbers of tokens, implicitly assuming that effective learning must be token-intensive. We revisit this assumption in the on-policy distillation (OPD) setting, which naturally admits dense teacher supervision at every generated token. Using the Qwen3 family, we discover a counter-intuitive phenomenon: reasoning can be effectively incentivized by an extremely small fraction of generated tokens--as few as one or two tokens per reasoning trajectory, corresponding to only 0.05% of all tokens. Surprisingly, this sparse supervision in most cases matches or surpasses full-token training in improving reasoning ability, despite excluding the vast majority of generated tokens from the training objective. This phenomenon is consistently observed across nine teacher--student configurations spanning different model scales on mathematical reasoning tasks, and is further validated on coding reasoning, Llama models and Proximal Policy Optimization (PPO)-based reinforcement learning with verifiable reward (RLVR). Interestingly, such extremely sparse supervision may be closer to the natural learning process: rather than correcting every step word by word, one reflects on a few critical reasoning steps, updates prior understanding, and continues the trial-and-error, avoiding micro-level corrections while remaining remarkably effective. Overall, our results challenge the assumption that effective post-training must be token-intensive and point to a new direction for understanding and designing more efficient post-training algorithms.", "url": "https://wpnews.pro/news/extremely-sparse-supervision-incentivizes-reasoning-ability", "canonical_source": "https://www.machinebrief.com/news/extremely-sparse-supervision-incentivizes-reasoning-ability-qu30", "published_at": "2026-09-07 04:00:00+00:00", "updated_at": "2026-09-07 04:56:17.714829+00:00", "lang": "en", "topics": ["large-language-models", "ai-research"], "entities": ["arXiv", "Qwen3", "Llama"], "alternates": {"html": "https://wpnews.pro/news/extremely-sparse-supervision-incentivizes-reasoning-ability", "markdown": "https://wpnews.pro/news/extremely-sparse-supervision-incentivizes-reasoning-ability.md", "text": "https://wpnews.pro/news/extremely-sparse-supervision-incentivizes-reasoning-ability.txt", "jsonld": "https://wpnews.pro/news/extremely-sparse-supervision-incentivizes-reasoning-ability.jsonld"}}