BenchAudit – Auditing LLM benchmarks for leaks, data reuse, and harness gaps BenchAudit launched a benchmark-integrity auditing tool that scores LLM benchmarks on four 10-point dimensions — Exposure, Provenance, Freshness and Harness — and rolls them into an Overall score out of 10 with letter tiers from S (85–100) down to D (0–29). The tool flags benchmarks whose test prompts and answers leak through public downloads, GitHub repos or auto-approved click-through gates, and separates private held-out tests that stay trustworthy with age from public datasets that lose value over time. BenchAudit also publishes cross-study insights comparing integrity scores against live official model leaderboards, excluding frozen legacy leaderboards that lack recent frontier models. All TiersTier S 85–100 Tier A 70–84 Tier B 50–69 Tier C 30–49 Tier D 0–29 Domains Skills multi-select Showing 0 benchmarks Benchmark & Hub Category & Domains Exposure /10↕iEvaluates whether test prompts and answers are kept strictly private, or exposed via public downloads, GitHub repos, or auto-approved click-through gates. Provenance /10↕iEvaluates how much of the benchmark was newly authored from scratch versus reused from older public exams, competition archives, or web corpora. Freshness /10↕iEvaluates how recently tasks were refreshed relative to exposure risk, including rolling post-cutoff updates and leaked-task pruning. Harness /10↕iEvaluates whether models run in an isolated offline sandbox scored by an independent evaluator, free from live-internet or container/git leaks. Overall /10▼ Tier Details No benchmarks match the current filters Try clearing one or more Domain, Skill, Tier, or Search filters. Reset All Filters Insights Cross-study findings connecting benchmark integrity scores with live official model leaderboards, and more Syncing standings… 1. Fortified Leaders vs. Exposed Over-Performers Compares current-generation models across actively updated Tier S held-out evaluations and active Tier C & D benchmarks. Frozen legacy leaderboards missing recent frontier models are excluded to prevent cohort bias. Each shape is one audited benchmark: left to right shows how many months old it is since its last major update, and bottom to top shows its integrity score 0–100 . Circles are private held-out tests that stay trustworthy with age; diamonds are public or easy-to-access datasets that tend to leak and lose value over time. Hover or click any shape to inspect it. Distribution of Tiers S–D and average scores across fields. Click any bar to filter the table above. 9 Domains4 Categories 4. Access Barrier Effectiveness Groups benchmarks by how strictly their questions and answers are protected—from locked private servers at the top to open public downloads at the bottom. Each bar shows that group's average integrity score out of 100 , its lowest-to-highest range, and clickable benchmark examples.