cd /news/ai-safety/gauntletscore-auditable-fact-checkin… · home › topics › ai-safety › article
[ARTICLE · art-171558] src=machinebrief.com ↗ pub= topic=ai-safety verified=true sentiment=↑ positive

GauntletScore: Auditable Fact-Checking That Matches the Best Published Checkers on LLM-AggreFact

GauntletScore, an evidence engine that queries independent readers from different providers and combines their answers with a fixed Bayesian rule, reached 79.51 balanced accuracy on the full LLM-AggreFact test split of 29,320 claims across 11 datasets, according to the arXiv paper 2611.22027v1. That score is above every average published on the leaderboard as of 15 November 2026 and level with the top system, nimble-v4-max at 79.45, whose score falls inside GauntletScore's 95% interval of 78.47 to 80.53. On AggreFact-CNN, full-length news articles checked against multi-sentence summaries, GauntletScore scored 76.45, the best on the leaderboard, against 70.1 for the next best and 63.9 to 67.8 for the six systems with the highest averages; the authors release per-claim predictions and the scoring script.

recommendation

Route fact-check verdicts through multiple independent reader models and flag a contradiction only when two readers agree; the ensemble beat two of its three readers alone by 1.66 and 2.17 points on LLM-AggreFact.

more in playbook: Verifying model output
by read1 min views2 publishedNov 18, 2026

arXiv:2611.22027v1 Announce Type: new Abstract: Checking whether a generated sentence is supported by its source document is a standard safeguard for AI-written text, and the strongest published checkers are single models. We describe GauntletScore, an evidence engine whose few rules and thresholds were set on the benchmark's development split. It puts typed questions to independent readers from different providers and records each answer as a measurement in an evidence ledger. It combines the measurements with a fixed Bayesian rule that counts a contradiction only when two readers agree. On the full LLM-AggreFact test split (29,320 claims across 11 datasets), following the leaderboard's approach, GauntletScore achieves 79.51 balanced accuracy, above every average published on the leaderboard as of 15 November 2026. That places it level with the best of them, nimble-v4-max (79.45 from its published dataset scores; the engine's 95% interval, 78.47 to 80.53, contains it). On the same claims it beats two of its three readers used alone (Readers A and B) by 1.66 and 2.17 points (95% intervals 0.80 to 2.53 and 1.29 to 3.06). Without AggreFact-CNN, described below, the leads are 0.90 and 1.32 and both intervals stay above zero. The engine also records the evidence behind every verdict, which a single model does not. Its largest margin is on full-length news articles checked against multi-sentence summaries (AggreFact-CNN), a format that resembles much regulated writing. There it scores 76.45, the best on the leaderboard, against 70.1 for the next best and 63.9 to 67.8 for the six systems with the highest averages. The test split was run once, on an engine frozen before the first test claim, under a stopping plan committed in advance. We release per-claim predictions and the scoring script.

── more in #ai-safety 4 stories · sorted by recency
── more on @gauntletscore 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/gauntletscore-audita…] indexed:0 read:1min 2026-11-18 · —