Artificial Analysis Index v5 goes 60% private; Opus 5.5 leads at 58

Artificial Analysis Index v5 now draws roughly 60% of its test sets from private tasks, and Anthropic's Opus 5.5 leads the ranking at a score of 58, according to Artificial Analysis. The new tasks grade mergeable code and terminal science work, and the open-weights gap widened to 12 points while cost per task spans more than 13x across models. Artificial Analysis reports that models topping DeepSWE land mid-pack on the index, where graders ask whether a maintainer would merge the change.

Most of the test sets are now private, and new tasks grade mergeable code and terminal science work, so rankings are harder to game. The open-weights gap widened to 12 points, and cost per task now spans more than 13x across models. Read: Most of the test sets are now private, and new tasks grade mergeable code and terminal science work, so rankings are harder to game. The open-weights gap widened to 12 points, and cost per task now spans more than 13x across models. Read: Models that top DeepSWE land mid-pack here, where prompts leave gaps and graders ask whether a maintainer would merge the change. A high DeepSWE rank alone says little about handling vague real tickets, and cost per task is reported beside each score. Read: Artificial Analysis scores it 44, up from 41, but it spends 145k output tokens and $1.49 per task, about 40% more output than before. Its low per-token price does not carry through to the bill, so cost per task is the figure that decides its value. Watch: Gemini turns plain-language requests into actions, plans events and tracks scene state while the world renders in real time, so an LLM acts as the director of a simulator. Access, pricing and benchmarks are not stated, so the demo is the only evidence so far. Read: It ships cost data and a forkable repo for a long-running multi-agent harness that enforces brand rules with code gates and runs subagents asynchronously. It deletes workaround rules as models improve, so the harness gets simpler over time instead of piling up patches. Read: Accuracy rose from 81.9% at similar cost, while walking the taxonomy top-down with beam search failed. Classifying into a large taxonomy is a common agent pipeline step, so this gain comes from changing the pipeline shape, not from paying for a bigger model.