{"slug": "when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify", "title": "When Is Proof-Step Verification Worth Its Cost? Operating Requirements of Certify-or-Decline Verification for Open-Weight Mathematical Reasoning", "summary": "A new arXiv paper (2611.14794v1) evaluating certify-or-decline proof-step verification on open-weight models found that certification raised answer-key-match precision by 31 percentage points on the strong GPQA configuration and 46 points on AIME-2025, but only answered 35% and 21% of attempts respectively. On near-ceiling MATH-500, gains were at most 5 points at a loss of 13-27 points of coverage, and an output-token-matched self-consistency selector reached 82.9% precision at 35% coverage versus the verifier's pooled 87.6% (paired difference +4.8 points, 95% interval [-8.2, 18.5]). The authors report that 52 of 88 completed problems ended in invalid proof construction without ever reaching a judge, and that external-LLM acceptance of rendered certified chains was 61.0-67.6% against 87.6% answer-key match, motivating separate reporting of execution failures, selected-answer precision, and independently assessed proof validity.", "body_md": "arXiv:2611.14794v1 Announce Type: new \nAbstract: Proof-step verification rewrites a solver's answer into typed steps and uses LLM judges to certify or decline. We evaluate this pipeline on open-weight models across GPQA, AIME-2025, and a MATH-500 subset. Certification raises answer-key-match precision by 31 percentage points on the strong GPQA configuration and 46 on AIME, while answering only 35% and 21% of attempts. On near-ceiling MATH-500, gains are at most 5 points at a loss of 13-27 points of coverage. An output-token-matched self-consistency selector obtains 82.9% precision at 35% coverage, compared with the verifier's pooled 87.6% (paired difference: +4.8 points, 95% interval [-8.2, 18.5]). A diagnostic sample reveals a distinct failure mode: 52 of 88 completed problems end in invalid proof construction without ever reaching a judge. Finally, external-LLM acceptance of rendered certified chains is 61.0-67.6%, compared with 87.6% answer-key match. These audits lack human or formal-kernel ground truth. Open-weight auditor pairs show near-zero chance-corrected agreement, which measures consistency rather than accuracy against proof truth. These results motivate separate reporting of execution failures, selected-answer precision, and independently assessed proof validity. We provide per-configuration results and evaluation details in the supplement.", "url": "https://wpnews.pro/news/when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify", "canonical_source": "https://arxiv.org/abs/2611.14794", "published_at": "2026-11-14 05:00:00+00:00", "updated_at": "2026-11-14 05:22:08.329581+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-safety"], "entities": ["arXiv", "GPQA", "AIME-2025", "MATH-500"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify", "markdown": "https://wpnews.pro/news/when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify.md", "text": "https://wpnews.pro/news/when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify.txt", "jsonld": "https://wpnews.pro/news/when-is-proof-step-verification-worth-its-cost-operating-requirements-of-certify.jsonld"}}