{"slug": "gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments", "title": "Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments", "summary": "Researchers introduced Gaia2, a benchmark for evaluating large language model agents in dynamic, asynchronous environments where scenarios evolve independently of agent actions. Testing of state-of-the-art models revealed that no single model excels across all capabilities, with GPT-5 achieving the highest overall score of 42% pass@1 but failing on time-sensitive tasks, while Claude-4 Sonnet trades accuracy and speed for cost and Kimi-K2 leads open-source models with 21% pass@1. The benchmark, built on the open-source Agents Research Environments platform, aims to provide infrastructure for developing and training practical agent systems that can handle temporal constraints, noisy events, ambiguity, and multi-agent collaboration.", "body_md": "# Computer Science > Artificial Intelligence\n\n[Submitted on 12 Feb 2026]\n\n# Title:Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environments\n\n[View PDF](/pdf/2602.11964)\n\nAbstract:We introduce Gaia2, a benchmark for evaluating large language model agents in realistic, asynchronous environments. Unlike prior static or synchronous evaluations, Gaia2 introduces scenarios where environments evolve independently of agent actions, requiring agents to operate under temporal constraints, adapt to noisy and dynamic events, resolve ambiguity, and collaborate with other agents. Each scenario is paired with a write-action verifier, enabling fine-grained, action-level evaluation and making Gaia2 directly usable for reinforcement learning from verifiable rewards. Our evaluation of state-of-the-art proprietary and open-source models shows that no model dominates across capabilities: GPT-5 (high) reaches the strongest overall score of 42% pass@1 but fails on time-sensitive tasks, Claude-4 Sonnet trades accuracy and speed for cost, Kimi-K2 leads among open-source models with 21% pass@1. These results highlight fundamental trade-offs between reasoning, efficiency, robustness, and expose challenges in closing the \"sim2real\" gap. Gaia2 is built on a consumer environment with the open-source Agents Research Environments platform and designed to be easy to extend. By releasing Gaia2 alongside the foundational ARE framework, we aim to provide the community with a flexible infrastructure for developing, benchmarking, and training the next generation of practical agent systems.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nPapers with Code\n\n*(*[What is Papers with Code?](https://paperswithcode.com/))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments", "canonical_source": "https://arxiv.org/abs/2602.11964", "published_at": "2026-06-07 01:36:17+00:00", "updated_at": "2026-06-07 01:46:20.695299+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "ai-research", "artificial-intelligence", "machine-learning"], "entities": ["Gaia2", "GPT-5", "Claude-4 Sonnet", "Kimi-K2", "Agents Research Environments"], "alternates": {"html": "https://wpnews.pro/news/gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments", "markdown": "https://wpnews.pro/news/gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments.md", "text": "https://wpnews.pro/news/gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments.txt", "jsonld": "https://wpnews.pro/news/gaia2-benchmarking-llm-agents-on-dynamic-and-asynchronous-environments.jsonld"}}