{"slug": "gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers", "title": "GauntletScore: Auditable Fact-Checking That Matches the Best Published Checkers on LLM-AggreFact", "summary": "GauntletScore, an evidence engine that queries independent readers from different providers and combines their answers with a fixed Bayesian rule, reached 79.51 balanced accuracy on the full LLM-AggreFact test split of 29,320 claims across 11 datasets, according to the arXiv paper 2611.22027v1. That score is above every average published on the leaderboard as of 15 November 2026 and level with the top system, nimble-v4-max at 79.45, whose score falls inside GauntletScore's 95% interval of 78.47 to 80.53. On AggreFact-CNN, full-length news articles checked against multi-sentence summaries, GauntletScore scored 76.45, the best on the leaderboard, against 70.1 for the next best and 63.9 to 67.8 for the six systems with the highest averages; the authors release per-claim predictions and the scoring script.", "body_md": "arXiv:2611.22027v1 Announce Type: new \nAbstract: Checking whether a generated sentence is supported by its source document is a standard safeguard for AI-written text, and the strongest published checkers are single models. We describe GauntletScore, an evidence engine whose few rules and thresholds were set on the benchmark's development split. It puts typed questions to independent readers from different providers and records each answer as a measurement in an evidence ledger. It combines the measurements with a fixed Bayesian rule that counts a contradiction only when two readers agree. On the full LLM-AggreFact test split (29,320 claims across 11 datasets), following the leaderboard's approach, GauntletScore achieves 79.51 balanced accuracy, above every average published on the leaderboard as of 15 November 2026. That places it level with the best of them, nimble-v4-max (79.45 from its published dataset scores; the engine's 95% interval, 78.47 to 80.53, contains it). On the same claims it beats two of its three readers used alone (Readers A and B) by 1.66 and 2.17 points (95% intervals 0.80 to 2.53 and 1.29 to 3.06). Without AggreFact-CNN, described below, the leads are 0.90 and 1.32 and both intervals stay above zero. The engine also records the evidence behind every verdict, which a single model does not. Its largest margin is on full-length news articles checked against multi-sentence summaries (AggreFact-CNN), a format that resembles much regulated writing. There it scores 76.45, the best on the leaderboard, against 70.1 for the next best and 63.9 to 67.8 for the six systems with the highest averages. The test split was run once, on an engine frozen before the first test claim, under a stopping plan committed in advance. We release per-claim predictions and the scoring script.", "url": "https://wpnews.pro/news/gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers", "canonical_source": "https://www.machinebrief.com/news/gauntletscore-auditable-fact-checking-that-matches-the-best-lza5", "published_at": "2026-11-18 05:00:00+00:00", "updated_at": "2026-11-18 05:15:55.054911+00:00", "lang": "en", "topics": ["ai-safety", "ai-research", "large-language-models", "artificial-intelligence"], "entities": ["GauntletScore", "LLM-AggreFact", "nimble-v4-max", "AggreFact-CNN", "arXiv"], "also_reported_by": [], "takeaway": "Combining independent reader models with a fixed Bayesian rule that counts a contradiction only when two readers agree reached 79.51 balanced accuracy on LLM-AggreFact, beating two of its three readers used alone by 1.66 and 2.17 points.", "recommendation": {"advice": "Route fact-check verdicts through multiple independent reader models and flag a contradiction only when two readers agree; the ensemble beat two of its three readers alone by 1.66 and 2.17 points on LLM-AggreFact.", "playbook": "output-verification", "playbook_url": "https://wpnews.pro/playbooks/output-verification"}, "alternates": {"html": "https://wpnews.pro/news/gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers", "markdown": "https://wpnews.pro/news/gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers.md", "text": "https://wpnews.pro/news/gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers.txt", "jsonld": "https://wpnews.pro/news/gauntletscore-auditable-fact-checking-that-matches-the-best-published-checkers.jsonld"}}