Confidence-Calibrated Inference Expansion for Evaluator-Guided Test-Time Reasoning

A compute-aware three-action controller for evaluator-guided test-time reasoning improved observed utility over calibration-selected fixed policies while cutting normalized online compute by about 30% on held-out width expansion and 16% on revision tasks, according to the arXiv paper 2610.20836v1. The controller scores terminal actions by predicted answer success and expansion by a rollout-based continuation-success surrogate, with compute cost priced explicitly, and learns action-specific confidence estimates with local uncertainty penalties for ambiguous decisions. Across four Qwen3-8B seeds, compute was lower than fitted Q in both settings while utility differences were small and mixed, framing evaluator-guided reasoning as resource allocation under uncertainty rather than a uniformly stronger reasoner.

arXiv:2610.20836v1 Announce Type: new Abstract: Evaluator-guided reasoning presents a local control problem at every step: trust the current answer, return a fallback, or spend more inference. We formulate this choice as a compute-aware three-action controller, scoring terminal actions by predicted answer success and expansion by a rollout-based continuation-success surrogate, with compute cost priced explicitly. A local action-gap analysis characterizes when score errors can change a decision, while an expected regret bound accounts for errors along the realized expansion path. Guided by this analysis, we learn and calibrate action-specific confidence estimates, with local uncertainty penalties for ambiguous decisions. On held-out width expansion and revision tasks, the controller improves observed utility over calibration-selected fixed policies while reducing normalized online compute by about 30% and 16%, respectively. Across four Qwen3-8B seeds, compute is lower than fitted Q in both settings, while utility differences are small and mixed. The results frame evaluator-guided reasoning as resource allocation under uncertainty, rather than a uniformly stronger reasoner.