A new benchmark called Humanity's Last Exam, designed to stump frontier AI models, is now available as a 10-question randomized quiz called The Adult SAT. The quiz lets users compare their scores agai…
The Center for AI Safety, with world experts, created Humanity's Last Exam (HLE), a benchmark of over 2,500 expert-level questions across disciplines to test AI reasoning. Even top models like GPT, Ge…
Two AI judge models scored an agent's answer 0.85 based solely on final answer matching, but the agent never retrieved the required Confluence page, leading to a trace-based score of 0.000. This expos…
Google has made Gemini 3.5 Flash generally available, offering faster performance and improved agentic and coding benchmarks over its predecessor, but developers migrating from the preview version mus…
Andon Labs co-founders Lukas Petersson and Axel Backlund are stress-testing frontier AI models by giving them real-world control over inventory, wallets, and physical storefronts, revealing unexpected…