{"slug": "premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews", "title": "PremiseCheck: Efficient Fact-Checking of Human and AI Peer Reviews", "summary": "Researchers introduced ReviewFact, a benchmark that labels factual errors in peer-review comments across NLP, machine learning, and the natural sciences while measuring cost and latency alongside accuracy, and PremiseCheck, a framework pairing an LLM that rewrites each comment into explicit premises without reading the paper with a decision model that checks each premise against the paper. PremiseCheck reaches accuracy comparable to GPT-5.5 across all three fields at about half its cost when each comment is checked on its own, and on NLP reviews at a quarter of the cost and less than half the latency, according to the arXiv:2610.21046v1 paper. The work targets peer review strained by growing submission volume, where human- or AI-written review comments can misstate what a paper says.", "body_md": "arXiv:2610.21046v1 Announce Type: new \nAbstract: Peer review is under strain from the growth of submissions, and review comments written by humans or AI can misstate what a paper says. We introduce ReviewFact, a benchmark that labels such factual errors in review comments under one definition across NLP, machine learning, and the natural sciences, and that measures cost and latency alongside accuracy. A frontier LLM checks comments well but must read the whole paper for each check, while cheap decision models, which score whether a statement holds in a text, fall far short on whole comments that mix several implicit assertions. PremiseCheck is a framework that combines the two. An LLM rewrites each comment as explicit premises without reading the paper, and a decision model checks each premise against the paper. When each comment is checked on its own, PremiseCheck reaches accuracy comparable to GPT-5.5 across all three fields at about half its cost, and on NLP reviews at a quarter of its cost and less than half its latency.", "url": "https://wpnews.pro/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews", "canonical_source": "https://www.machinebrief.com/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-re-wz5a", "published_at": "2026-10-16 04:00:00+00:00", "updated_at": "2026-10-16 05:08:49.645613+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "natural-language-processing", "machine-learning", "ai-safety"], "entities": ["ReviewFact", "PremiseCheck", "GPT-5.5", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews", "markdown": "https://wpnews.pro/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews.md", "text": "https://wpnews.pro/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews.txt", "jsonld": "https://wpnews.pro/news/premisecheck-efficient-fact-checking-of-human-and-ai-peer-reviews.jsonld"}}