Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Models

A new arXiv paper (2609.20846v1) reports that fine-tuning several 4B large reasoning models with a novel GRPO reward that encourages efficient reasoning about whether a task contains enough information improves abstention performance by 12.8% on average while shortening chains of thought by 44% on average. The authors compared large reasoning model behavior against a human study, finding that human reasoning effort on unanswerable tasks is upper-bounded by answerable tasks, whereas the models generate longer chains of thought on unanswerable than on answerable prompts. The reward is inspired by a resource-rational perspective on human cognition and retains the models' answering capabilities.

arXiv:2609.20846v1 Announce Type: new Abstract: While modern large reasoning models LRMs excel at providing correct answers in many tasks, we provide additional evidence for the observation that they often struggle with a critical capability: knowing when to abstain from answering. We analyze this gap by comparing LRM behavior to results from a human study, revealing that human reasoning effort on unanswerable tasks is upper-bounded by answerable tasks, whereas LRMs waste computational resources by generating longer Chains of Thought CoTs on unanswerable than on answerable prompts. To overcome this inefficiency, we take inspiration from a resource-rational perspective on human cognition and introduce a novel GRPO reward that encourages efficient reasoning about whether the task contains all the information needed to solve it. Fine-tuning several 4B LRMs with this reward leads to human-like abstention performance gains +12.8% on average while retaining answering capabilities and boosting the models' efficiency 44% shorter CoTs on average .