ECE achieves 97.8% accuracy on answered claims
Which summary reads better? Pick one — models revealed after.Both summaries are AI-generated.
Adding an "uncertain" abstention verdict to a tool-using fact-checking agent pushed selective accuracy on answered claims to 97.8% by deferring just 6 of 95 cases—almost all concentrated in weak-evidence settings—while overall accuracy stayed at 91.6%. Notably, this abstention gate didn't improve aggregate calibration metrics (ECE, Brier, AURC), so if you're routing verification agents in production, treat abstention as a targeted safety valve for epistemically thin evidence rather than a general confidence-calibration fix.
Implementing the Evidence Chain Evaluation framework for tool-using agents yields a 97.8% selective accuracy on fact-checking tasks by enabling the agent to abstain on uncertain claims instead of forcing a binary verdict. For production systems, this provides a reliable safety-valve mechanism that filters out weak or inconsistent source evidence by deferring low-reliability queries, even though it does not improve overall aggregate calibration metrics like the Brier score. This allows you to deploy highly reliable automated content verification workflows where the vast majority of claims are handled with near-perfect precision while ambiguous edge cases are safely escalated.