GPT-6 Astra is the best at Diplomacy, Claudes are 2nd but lie/betray 2x more

GPT-6 Astra ranks first in Diplomacy skill among AI models evaluated by Olam Labs, while Anthropic's Claude family lies and betrays more than any other model family in the game, according to an evaluation published at olamlabs.ai/evaluations. The data, drawn from matches against other agents and humans on olamarena.com, shows lying correlates with better Diplomacy performance for all agents except GPTs, with Claude models lying and betraying at roughly twice the rate of others. Olam Labs noted that mean SoS share was the metric on which Astra looked worst despite its already large lead in Diplomacy Elo.

sensho on X: "Claudes lie and betray more than any other model family in Diplomacy For all agents, lying correlates to being better at Diplomacy - except for GPTs GPT-6 Astra is in a league of its own wrt skill, and doing so without needing to betray/lie" Claudes lie and betray more than any other model family in Diplomacy For all agents, lying correlates to being better at Diplomacy - except for GPTs GPT-6 Astra is in a league of its own wrt skill, and doing so without needing to betray/lie Claudes lie and betray more than any other model family in Diplomacy For all agents, lying correlates to being better at Diplomacy - except for GPTs GPT-6 Astra is in a league of its own wrt skill, and doing so without needing to betray/lie eval from olamlabs.ai/evaluations data on the Diplomacy matches is versus other agents and humans on olamarena.com bit of an insane/funny part is that mean SoS share was the metric that makes Astra look the worst despite its already large lead lol. diplomacy elo