All articles August 27, 2026
Agnes AI's Agnes 2.5 Pro Beta scores 49 on the Artificial Analysis Intelligence Index, up 9 points from Agnes 2.5 Pro Alpha, driven by large agentic gains but using ~2x the output tokens
See model page Agnes AI is a Singapore-based AI lab that trains full-modality foundation models and offers them through a free omni-modal API. Agnes 2.5 Pro Beta is a beta checkpoint of its next 2.5 Pro release, a text, image, and video input reasoning model with text output.
At 49 on the Intelligence Index, Agnes 2.5 Pro Beta moves Agnes from mid-pack to the frontier-adjacent tier, just behind Gemini 3.5 Flash (high, 52) and GPT-5.6 Luna (max, 52) and ahead of MiniMax-M3 (45).
Key results:
➤ Agnes 2.5 Pro Beta scores 49 on the Artificial Analysis Intelligence Index, a 9-point jump from Agnes 2.5 Pro Alpha (40). This places it just behind Gemini 3.5 Flash (high, 52) and GPT-5.6 Luna (max, 52), and ahead of MiniMax-M3 (45).
➤ Agentic capabilities drive the jump. The Artificial Analysis Agentic Index rises from 25 to 44, just behind Gemini 3.7 Flash (high, 45) and ahead of Gemini 3.5 Flash (high, 40) and MiniMax-M3 (36). τ³-Banking nearly triples from 12% to 36%, and GDPval-AA v2 rises from an Elo of 1171 to 1456 against a human baseline of 1000.
➤ Frontier reasoning evaluations improves more modestly. Humanity's Last Exam rises from 34% to 38%, GPQA Diamond from 88% to 91%, and CritPt from 11% to 16%.
➤ The AA-Omniscience improvement from -25 to -11 comes from abstention, not increased accuracy. Agnes 2.5 Pro Beta attempts only 45% of questions against Agnes 2.5 Pro Alpha's 94%, cutting the hallucination rate from 88% to 33%, but AA-Omniscience Accuracy halves from 33% to 17%.
➤ The intelligence gain required ~2x as many tokens from its predecessor. Agnes 2.5 Pro Beta uses 50k output tokens per Intelligence Index task, more than double Agnes 2.5 Pro Alpha's 24k.
Additional model details:
➤ Context window: 1M tokens
➤ Max output tokens: 65k
➤ Input modalities: Text and image
➤ Pricing: $0.10 / $0.30 / $0.01 per 1M input / output / cache hit tokens
➤ Availability: Agnes AI first-party API
Agnes 2.5 Pro Beta's largest gains are on agentic work. It achieves an Artificial Analysis Agentic Index score of 44, up from 25 from its predecessor. It scores an Elo of 1456 on GDPval-AA v2 against a human baseline of 1000, up from 1171 for Agnes 2.5 Pro Alpha, ahead of MiniMax-M3 (1384) and just behind GPT-5.5 (xhigh, 1489) and Gemini 3.7 Flash (high, 1527).
Agnes 2.5 Pro Beta's AA-Omniscience score improves from -25 to -11, however the improvement comes from abstention rather than accuracy improvement. It attempted only 45% of questions compared to 94% for Agnes 2.5 Pro Alpha, and while its hallucination rate improves from 88% to 33%, its AA-Omniscience Accuracy halves from 33% to 17%.
Agnes 2.5 Pro Beta uses 50k output tokens per Artificial Analysis Intelligence Index task, more than double Agnes 2.5 Pro Alpha's 24k. It uses more tokens than Qwen3.8 27B (xhigh, 47k) and GLM-5.3 (max, 41k).
Full results across the Artificial Analysis Intelligence Index:
Read the latest
Intelligence at pocket scale: Benchmarking small models and mobile phones
Independent intelligence benchmarking of small language models on a set of evaluations chosen for mobile device use, launched alongside mobile phone inference benchmarking with Liquid AI. We evaluate the same quantized builds used on mobile phones, and performance is measured on real devices.
August 24, 2026
Announcing the Speech Agent Arena: Compare Speech agents in real world conversations
Announcing our new Speech Agent Arena, evaluating Speech to Speech models on real-world scenarios to analyze conversational preference and task success rate
August 24, 2026
Announcing the Artificial Analysis Search Index: Same Agent, Different Search
The Artificial Analysis Search Index benchmarks how search API providers perform on quality, cost, and speed when used by an agent. We compare different Search API providers across a series of search-related benchmarks using the same agentic setup.
August 18, 2026