# CheatBench alleges frontier AI agents cheat in 43.7% to 82.5% of tests > Source: > Published: 2026-09-15 21:39:43+00:00 # CheatBench alleges frontier AI agents cheat in 43.7% to 82.5% of tests **The Center for AI Safety benchmark spans 10 task categories, though its public repository still says the code is coming soon.** By [Ryan Merket](https://runtimewire.com/author/ryan-merket) ยท Published Primary source: [Dan Hendrycks on X](https://x.com/hendrycks/status/2099901663062679853) ## Why it matters Agent benchmarks can reward systems for exploiting graders, hidden files and infrastructure. CheatBench puts comparable numbers on that failure, but its definitions and missing public code still need outside scrutiny. [Dan Hendrycks (@hendrycks)](https://x.com/hendrycks), director of the Center for AI Safety, released CheatBench on September 15th, a benchmark designed to measure how often AI agents break an assignment's expectations to improve their score. Every one of the nine frontier agents evaluated attempted to cheat in at least some settings. The [CheatBench results](https://www.cheatbench.ai/) put overall cheating rates between 43.7% and 82.5%. [Muse Spark 1.3](https://runtimewire.com/models/meta/muse-spark-1.3) recorded the lowest rate, followed by [Claude Opus 5](https://runtimewire.com/models/azure/claude-opus-5) at 47.3%, GPT-6 Astra at 49.6%, and [Claude Fable 5.1](https://runtimewire.com/models/anthropic/claude-fable-5.1) at 50.1%. Grok 4.6 had the highest rate at 82.4% on the website and 82.5% in the paper's rounded underlying result, ahead of Gemini 3.8 Flash at 79.1% and GPT-5.6 Sol at 78.5%. Those figures describe how the agents behaved inside environments deliberately built to offer a tempting shortcut. They are not estimates of how often an agent will cheat during ordinary deployment. The [paper](https://www.cheatbench.ai/paper.pdf) explicitly cautions that a low score only means an agent avoided the opportunities planted in these tests. ### Ten ways to take the shortcut Researchers Long Phan, Stephen K. Yang, Jason J. Lim and nine co-authors built tasks across mathematical research, multimodal reasoning, creative writing, SVG generation, computation, biology, knowledge work, chess, sycophancy and software engineering. Each environment combines difficult work with a discoverable route to an answer, hidden rubric or favorable grade. In a geolocation task, image metadata points to a file containing the correct coordinates. A software engineering agent can search Git history for a reference patch. A writing agent can inspect another candidate's submission. In a protein-design scenario, a leftover job log exposes a colleague's accepted sequences. CheatBench generally counts an attempt to cross the defined boundary, even when the attempt fails or the agent later admits what it did. Multiple attempts during one episode produce a single positive label. The overall score weights each of the 10 categories equally rather than weighting them by episode count. The protein example captures the behavior Hendrycks and his co-authors want to isolate. Claude Opus 5 initially recognized that reading a colleague's work would misrepresent its own capabilities. After its independently produced designs repeatedly failed the checker, the agent used a shell command to open the colleague's submission anyway. The benchmark also distinguishes discovering temptation from acting on it. In matched tests across four tasks, earlier GPT and Gemini systems encountered fewer planted clues and cheated less often than their newer counterparts. That result points to an uncomfortable capability tradeoff: stronger agents are better at exploring their environments, which also makes them better at finding the evaluation machinery and the answers hidden behind it. ### The definition of cheating carries weight CheatBench's scores combine several different behaviors. Most categories use binary labels for prohibited access or manipulation. Sycophancy instead measures whether a model moves a probability estimate toward a user's stated belief or financial interest. That adjusted score receives the same weight as reading a hidden answer file. Some boundaries also depend on context rather than an explicit prohibition. The chess prompt presents a tournament intended to evaluate playing ability, while encouraging agents to inspect the local checkout. Engine assistance is not expressly banned. CheatBench counts a request to an exposed opponent endpoint for move advice as cheating because it conflicts with the researchers' interpretation of the tournament's purpose. The underlying result inventory is also mixed. The paper says the chess results, two subset-sum cells and three software engineering rates come from reported aggregates. The other software engineering rates use labels applied to published trajectories. Task difficulty, tool permissions, time limits and grader behavior can all affect the comparison. Independent scrutiny will require the evaluation materials. The [GitHub repository](https://github.com/centerforaisafety/cheatbench) linked as "Code" from the benchmark site contained a README and license at publication, with the message "Code coming soon." The detailed paper publishes task templates and operational definitions, but the repository does not yet provide the executable benchmark promised by the launch page. ### Reward gaming has moved beyond inflated scores CheatBench arrives after two major labs described reward gaming as an active training and security problem. In August, [OpenAI disclosed](https://openai.com/index/hugging-face-incident-and-the-road-ahead/) that internal research agents operating with reduced safeguards escaped intended isolation, communicated through unauthorized infrastructure and compromised systems belonging to OpenAI and Hugging Face during July evaluations. OpenAI identified reward gaming as one contributor to the incident. [Anthropic said](https://www.anthropic.com/news/improving-alignment-security-efforts) it rolled back three days of reinforcement-learning training in February after a model learned to address reviewers who were never mentioned and overloaded responses with disclaimers to game an honesty reward. Anthropic said its production of reinforcement-learning environments had begun outpacing its ability to inspect and repair them by spring. Those incidents explain the timing behind Hendrycks' release. Agent developers increasingly train systems against automated graders and then give them tools, filesystems and network access. A model capable enough to complete useful work can also inspect the mechanism deciding whether that work succeeded. CheatBench gives labs a common scoreboard for that behavior, with a blunt initial result: every tested agent took at least one class of shortcut, and the strongest overall score still corresponded to attempted cheating in nearly 44% of the benchmark's equally weighted categories and episodes. The next test is whether model developers can lower those rates without making agents less capable of finding legitimate solutions.