SchemeArena: Factorized Stress Testing of Scheming in LLM Agents

A new benchmark called SchemeArena factorizes stress testing of scheming in LLM agents, examining how covert pursuit of misaligned goals arises from the interaction of instrumental goals, environmental affordances, oversight conditions, and perceived consequences. The work addresses a gap in prior research, which the authors state examines only a single factor at a time.

We study scheming in LLM agents, in which agents covertly pursue misaligned goals. Our focus is to understand how scheming arises from the interaction of key factors, such as instrumental goals, environmental affordances, oversight conditions, and perceived consequences. Prior work examines only a s