A macro execution approach reduced Codex rounds by up to 84% in a 60-task benchmark, according to Tura maintainer and open-source implementer. Testing Codex CLI 0.144.1 with GPT-5.6-sol on 60 DeepSWE …
DeepSWE, a new benchmarking platform for AI coding agents, achieves clearer separation between frontier models than older benchmarks like SWE-bench by using contamination-free tasks across 91 reposito…
A developer open-sourced a macro execution layer for coding agents that reduces the number of model turns by executing routine workflow steps together. On a 60-task DeepSWE benchmark, the macro approa…
Moonshot AI open-sourced Kimi K3, a 2.8-trillion-parameter Stable LatentMoE model with a 1M-token context window and native vision, claiming it as the first open 3-trillion-class model with weights du…
A developer proposes that Anthropic's Claude Code team implement a macro command to batch multi-step coding tasks (e.g., patching, building, testing) into a single turn, claiming it can reduce LLM tur…
Tura, an open-source coding agent, reduces LLM turns by 80% while achieving a higher success rate on DeepSWE v1.1 tasks. In 60 sessions with GPT-5.6 SOL, Tura's Direct configuration used 83.5% fewer t…
OpenAI reviewed the SWE-Bench Pro AI coding test and found roughly 30% of its tasks are flawed, leading the company to withdraw its endorsement. The problems include overly strict or vague task requir…
An evaluation of Anthropic's Sonnet 5 and Opus 4.8 models across 24 coding tasks shows Sonnet produces clearer and more intentional patches, while Opus yields simpler, more robust, and more minimal di…
OpenAI published research on using AI to help diagnose rare genetic diseases in children, emphasizing the model's role in narrowing diagnostic search space rather than replacing physicians. Anthropic …
DeepSWE v1.1 updates the benchmark for long-horizon engineering tasks with isolated verification and structured test reports, making results more reproducible and harder to game. Pass rates remain clo…
Fable 5 has achieved a performance level on par with GPT-5.5 in the Artificial Analysis Coding Agent Index, a composite benchmark measuring real-world coding agent performance across software engineer…
Researchers have found that Claude Opus was contaminated on approximately 12% of SWEBench Pro tasks, meaning the model may have encountered those benchmark problems in its training data. The contamina…
The cost of frontier proprietary models increased significantly in May 2026, with Gemini 3.5 Flash priced three times higher than its predecessor and GPT 5.5 costing double GPT 5.4, while open-weight …
A DeepSWE benchmark audit found that DeepSeek-v4-pro's reported 8% solve rate and $4.22 average cost are unreliable due to inflated pricing and procedural errors. The audit team solved all three tasks…
DeepSWE has demonstrated that a fully optimized GPT 5.5 model can generate more intelligence at a lower cost than a maxed-out Opus 4.8, marking a significant efficiency breakthrough in AI performance.…
Anthropic released Claude Opus 4.8, an incremental upgrade with improved agentic judgment and honesty, achieving 84% on Online-Mind2Web and roughly 4x fewer code flaw overlooks. Harvard's Zitnik Lab i…
OpenAI has acknowledged in its official documentation that its GPT-5.5 model exhibits "diminished intelligence" and reduced nuance in complex creative generation compared to its predecessor, validatin…
DataCurve released DeepSWE, a new benchmark for evaluating frontier coding agents on original, long-horizon software engineering tasks. The benchmark features contamination-free tasks written from scr…