{"slug": "inverse-reasoning-models-for-efficient-post-training-and-verification-from-only", "title": "Inverse Reasoning Models for Efficient Post-Training and Verification from Answer-Only Data", "summary": "A new arXiv paper (2610.18878v1) proposes an \"inverse reasoning LLM\" self-distillation technique that trains on answer-only data by treating chain-of-thought as latent variables, then reuses the inverse model across incoming batches for offline continual post-training. Using Qwen3-0.6B on Countdown and GSM8K, the authors report that after GRPO training saturates, a single offline distillation round roughly doubles pass@1 on hard prompts and outperforms rejection-sampling self-distillation on every hard-tail metric at every mix ratio on Countdown, with smaller gains on GSM8K. The same inverse model also serves as a verifier, raising best-of-n selection accuracy on Countdown from 0.753 under majority voting to 0.969.", "body_md": "arXiv:2610.18878v1 Announce Type: new \nAbstract: Post-training reasoning LLMs often requires making a tradeoff between costly supervision and compute inefficiency: supervised finetuning typically requires reasoning traces, which is data inefficient since in many applications one only has prompt-answer tuples and lacks the groundtruth chain-of-thought (CoT) sequence. On the other hand, reinforcement learning (with verifiable rewards, for example) can operate without CoT data but it is much more compute-inefficient since it requires multiple autoregressive completions per optimization step, none of which are guaranteed to give a non-zero policy gradient. In this work we explore a middleground, a self-distillation technique that uses answer-only data, treating CoT as latent variables. This relies on training a partner ``inverse reasoning LLM\" that generates the original's reasoning traces conditioned on the answer. The inverse model is trained once and reused across incoming batches of answer-only data, with all generation happening offline and up front, making the approach particularly suited to continual post-training. Using Qwen3-0.6B on the Countdown and GSM8K tasks, we show that after GRPO training has saturated, a single offline distillation round from the inverse model roughly doubles pass@1 on hard prompts and, on Countdown, outperforms rejection-sampling self-distillation on every hard-tail metric at every mix ratio (with smaller gains on GSM8K); rejection sampling by construction has no signal on problems the model never solves on its own. Continued GRPO on the fresh data retains an edge at hard pass@1 but trails distillation at pass@8 and pass@32 while regenerating rollouts at every step. As an added bonus we see that the inverse model has a second application as an accurate verifier of correctness. When used for best-of-$n$ selection at test time, it raises selection accuracy on Countdown from 0.753 (majority voting) to 0.969.", "url": "https://wpnews.pro/news/inverse-reasoning-models-for-efficient-post-training-and-verification-from-only", "canonical_source": "https://arxiv.org/abs/2610.18878", "published_at": "2026-10-15 04:00:00+00:00", "updated_at": "2026-10-15 05:50:44.448149+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["Qwen3-0.6B", "Countdown", "GSM8K", "GRPO", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/inverse-reasoning-models-for-efficient-post-training-and-verification-from-only", "markdown": "https://wpnews.pro/news/inverse-reasoning-models-for-efficient-post-training-and-verification-from-only.md", "text": "https://wpnews.pro/news/inverse-reasoning-models-for-efficient-post-training-and-verification-from-only.txt", "jsonld": "https://wpnews.pro/news/inverse-reasoning-models-for-efficient-post-training-and-verification-from-only.jsonld"}}