Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments

Researchers introduced Gaia2, a benchmark for evaluating large language model agents in dynamic, asynchronous environments where scenarios evolve independently of agent actions. Testing of state-of-the-art models revealed that no single model excels across all capabilities, with GPT-5 achieving the highest overall score of 42% pass@1 but failing on time-sensitive tasks, while Claude-4 Sonnet trades accuracy and speed for cost and Kimi-K2 leads open-source models with 21% pass@1. The benchmark, built on the open-source Agents Research Environments platform, aims to provide infrastructure for developing and training practical agent systems that can handle temporal constraints, noisy events, ambiguity, and multi-agent collaboration.