cd /news/artificial-intelligence/conserved-verdicts-drifting-commitme… · home › topics › artificial-intelligence › article
[ARTICLE · art-171525] src=arxiv.org ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Conserved Verdicts, Drifting Commitments: Identifying Harmful Overthinking

A new arXiv paper (2611.21946v1) shows that reasoning models can lose correct answers by continuing to think, and that final-correctness probes alone cannot decide when to stop. Measuring on MATH-500, the authors found that at all 379 points where DeepSeek-R1-Distill-Qwen-1.5B held a correct answer, continuing lost 3.6 points on average (95% CI [2.2, 5.2]), with losses concentrated in a few problems; across six reasoning models from three families on MATH-500 and OlympiadBench with budgets up to 16,384 tokens, declining accuracy curves came from which chains remained rather than from chains degrading. An oracle stopping rule gains at most +1.8 and +2.5 points at 1.5B and 7B, and the authors argue the framework defines what to measure before building a stopping rule.

recommendation

Measure a model's commitment (accuracy if stopped now) alongside its verdict before building a stopping rule; final-correctness probes alone cannot predict whether continuing helps or hurts.

more in playbook: Evals and testing for LLM apps
by read1 min views11 publishedNov 18, 2026

arXiv:2611.21946v1 Announce Type: new Abstract: A reasoning model that has reached a correct answer can lose it by continuing to think. Deciding whether to stop means comparing two probabilities at the same point of a chain: that the answer the model would give if stopped now is correct (its commitment), and that the answer it will finish with is correct (its verdict). We show that the verdict alone, which is what final-correctness probes estimate, cannot make this comparison. As a forecast of a fixed outcome it has no predictable drift, whether continuing helps or hurts. The commitment-verdict gap equals the expected remaining change in answer quality and the accuracy gained by stopping. A second identity splits any accuracy-versus-budget curve into answer change within chains and change in which chains are still running. We measure both on MATH-500. At all 379 points where DeepSeek-R1-Distill-Qwen-1.5B holds a correct answer, selected without regard to how its chain ends, continuing loses 3.6 points on average (95% CI [2.2, 5.2]), and the loss concentrates in a few problems. In six reasoning models from three families, on MATH-500 and OlympiadBench and with budgets up to 16,384 tokens, declining accuracy curves come from which chains remain, while the chains still reasoning keep or improve their answers. An oracle stopping rule gains at most +1.8 and +2.5 points at 1.5B and 7B. The framework says what to measure before building a stopping rule.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @deepseek-r1-distill-qwen-1.5b 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/conserved-verdicts-d…] indexed:0 read:1min 2026-11-18 · —