GPT-6 Astra and Claude Opus 5.5 face off in a social deception benchmark testing skepticism, bluffing, and multi-agent reasoning under deception.
How do you test whether an AI model can lie convincingly? #
One researcher built a benchmark around Blood on the Clock Tower, a social deception game similar to Werewolf or Mafia, and had frontier models play hundreds of matches against each other. The setup splits players into a good team and an evil team. Evil players know their teammates and coordinate in secret, including a lead “demon” who kills a player each night. Good players have no idea who’s on which side and must piece together the truth from testimony, partial information, and outright lies before the demon wipes them out. It’s a clean way to measure skepticism, coordination, and bluffing because there’s no way to fake your way through hundreds of games of multi-agent social reasoning without those skills actually being present.
TL;DR #
- GPT-6 Astra , released by OpenAI in September, is currently the strongest model tested at Blood on the Clock Tower, beating Claude Opus 4.6 in roughly 92% of head-to-head games according to the benchmark’s creator.
- The gap between the two models works out to about 430 ELO points in the benchmark’s rating system, a sizable margin for a task that depends on long-horizon reasoning and coordination rather than raw factual recall.
- OpenAI’s models improved fastest on the good side , not the evil side. The jump from GPT-5.4 to GPT-6 Astra was about 580 ELO points, enough that Astra wins around 97% of games against its predecessor.
- The core skill driving that improvement is skepticism. OpenAI’s newer models treat reported evidence as one data point among many, while Anthropic’s models were observed taking claims and outcomes more at face value.
- A documented example showed a “slayer” character’s missed shot (meant to be strong evidence of innocence) accepted immediately by Claude-controlled good teams, while GPT-controlled good teams held out, reasoned through the possibility of sabotage, and ultimately found the real demon.
- Scaling intelligence doesn’t obviously favor one side. As both teams got smarter, evil’s win rate dipped slightly, but not by a statistically meaningful amount, suggesting the benchmark resists saturation as new models arrive.
- Mixed teams of different models didn’t outperform single-model teams. A hybrid squad of Opus 5.5 and GPT-5.6 instances performed somewhat worse than teams built from either model alone, though the sample size was small.
demandsof enterprises.
What exactly is Blood on the Clock Tower measuring? #
The game forces two very different cognitive skill sets to run in parallel. Evil players need to construct a plausible shared story, keep their lies consistent with each other under cross-examination, adapt those lies as new information surfaces, and sometimes make strategic sacrifices (like offering up a teammate for execution) to protect a more valuable player. Good players need to pool incomplete and sometimes deliberately corrupted information, figure out who to trust, and reason collectively about which players are most implicated by overlapping evidence, all while accounting for the possibility that some of what they’re hearing is a lie.
This matters for people building with AI because it’s a rare setting where you can watch long-horizon coherence, theory of mind, and deception detection play out across many agents at once, rather than inferring those capabilities indirectly from static benchmark scores.
How much better is GPT-6 Astra than Claude Opus? #
Based on 368 games where one team was entirely one model and the opposing team was entirely another, GPT-6 Astra sits at the top of the leaderboard, ahead of Claude Opus 4.6 by about 430 ELO points, a gap that translates to roughly a 92% win rate for Astra in direct matchups. That ranking wasn’t fixed over the year. As recently as March, OpenAI’s GPT-5.4 trailed behind Anthropic’s Opus 4.6. The turnaround came specifically from OpenAI’s side: between GPT-5.4 and GPT-6 Astra, the benchmark creator measured a roughly 580 ELO point jump, strong enough that Astra wins about 97% of games against its own predecessor.
Why are OpenAI’s models winning now? #
The improvement isn’t evenly spread across both sides of the game. Astra’s advantage over older models is much larger when it’s playing on the good team than when it’s playing evil. The driver appears to be skepticism: OpenAI’s recent models are more willing to treat any single piece of evidence as provisional and weigh it against everything else that’s happened in the game, rather than accepting it at face value.
One documented comparison makes the difference concrete. In a scenario where a “slayer” character takes a shot at who they believe is the demon and the shot fails (normally strong evidence the target is innocent), Claude-controlled good teams immediately accepted the target as cleared. GPT-controlled good teams, given the identical setup, refused to treat the missed shot as conclusive, reasoned that the evil team might have sabotaged the slayer’s power on purpose, and worked out the correct explanation by the next day, leading them to execute the actual demon and win. The benchmark creator also tracked each good player’s private daily estimate of who the demon was likely to be, and found OpenAI’s models converged on the correct answer more accurately and faster as games progressed than Anthropic’s models did.
Does getting smarter help the liars or the truth-seekers? #
demandsof enterprises.
Passed
*✓*SOC 2 Type 2
*✓*HIPAA
*✓*CAIQ attestation
*✓*OCSF logging
*✓*Role-based access
*✓*Self-hosted
This was one of the open questions the benchmark was built to probe: if every model in a game gets more capable, does that favor good (because sharper reasoning finds the demon faster) or evil (because sharper reasoning makes better lies)? The data didn’t show a clear answer either way. Evil’s win rate ticked down slightly as both sides got stronger, but the effect was too small to distinguish from random variation. That’s actually a useful property for a benchmark meant to track progress over time: it suggests the game won’t become trivially easy or impossible as new frontier models arrive, since the difficulty bar is set by the other models you’re playing against, not by a fixed dataset.
Do mixed-model teams perform better than single-model teams? #
A smaller side experiment tested whether teams built from different models might benefit from a diversity of reasoning styles and blind spots, similar to how diverse human teams sometimes outperform homogeneous ones. A team split evenly between Opus 5.5 and GPT-5.6 played around 130 games against other top models. The mixed team performed somewhat worse than teams made up of either model alone, though the difference wasn’t large enough to be statistically significant. It’s a preliminary result based on a limited number of games, not a settled finding, but it pushes against the assumption that combining frontier models automatically improves collective performance.
Is this benchmark actually useful, or just a novelty? #
For people evaluating models on reasoning and coordination rather than pure benchmark scores, this kind of environment has a real advantage: it’s hard to game and hard to saturate. The game’s outcomes depend on sustained multi-turn reasoning across many agents, some of whom are actively trying to mislead the others, which is a very different skill from answering a static test question correctly. The tradeoff is that building and validating a fair game engine is expensive and time-consuming. The creator reported spending roughly $20,000 in API credits on the project, with a large share going toward iterating on the game’s underlying rules and probability tables to keep outcomes fair rather than skewed toward one side. Much of that design work, including running batches of test games and reviewing them for unrealistic or unbalanced outcomes, was reportedly handled by Claude operating with minimal human oversight, which is itself a notable data point about how far AI-assisted environment design has come.
Frequently Asked Questions #
What is Blood on the Clock Tower in the context of AI benchmarking?
It’s a social deception game, similar to Werewolf or Mafia, adapted into an automated benchmark where AI models play as either a good team trying to identify a hidden “demon” or an evil team trying to bluff, coordinate, and kill off the good team first.
Which model performed best in the benchmark?
GPT-6 Astra, released by OpenAI in September, currently ranks highest, beating Claude Opus 4.6 in about 92% of head-to-head games according to the benchmark’s ELO ratings.
Why do OpenAI’s models reportedly do better than Anthropic’s in this game?
The gap traces mainly to performance on the good team, where OpenAI’s models showed more skepticism toward individual pieces of evidence and weighed new information against the broader game state, while Anthropic’s models were observed accepting reported outcomes more readily.
Does this mean Claude models are worse at reasoning in general?
Not necessarily. The benchmark measures a specific combination of skepticism, deception detection, and multi-agent coordination under adversarial conditions. It doesn’t directly measure coding, math, or other reasoning domains where rankings could differ.
Can this benchmark be reproduced or extended?
The creator built the game engine and harness with AI assistance and made it available for others to run their own experiments, since the comparison covered only a fraction of the questions the environment could be used to test.