VS Code 1.135, released August 26, introduces the Agent Host Protocol (AHP), an open, MIT-licensed specification that decouples AI agent sessions from the editor window, allowing sessions to persist i…
SWE-bench remains the most widely used open-source benchmark for AI coding agents, with 2,294 tasks from 12 Python repositories, but newer benchmarks like Terminal-Bench, SWE-Bench Pro, and Senior SWE…
Anthropic's Claude Sonnet 5, priced at $2/$10 per million tokens during an introductory period (standard $3/$15 after August 2026), matches or nearly matches the flagship Claude Opus 4.8 on knowledge …
XAI released Grok 4.6 on August 7, a language model built on the same 1.5 trillion-parameter V9 foundation as Grok 4.5 but with improved post-training, including better supervised fine-tuning and rein…
Researchers introduced SWE-Touch, a benchmark evaluating coding agents' ability to repair software after users directly edit code, finding that most models' performance drops significantly when user e…
Alibaba released Qwen 3.8 Max, a 2.4 trillion parameter open-weight AI model with 95 billion active parameters, set to become the largest open-weight model once weights are open-sourced about a week a…
Medley, a harness system that orchestrates model workers with planning, routing, and acceptance stages, achieved the highest reported results on four public benchmarks: Terminal-Bench 2.1 (93.25% task…
Vercel's AI Gateway now offers Laguna S 2.1, an open-weight Mixture-of-Experts model from Poolside that supports up to 1M tokens and agentic coding tasks. The model achieves 70.2% on Terminal-Bench 2.…
OpenAI published an audit of SWE-Bench Pro on July 8, 2026, estimating that roughly 30% of its tasks are broken, which undermines the assumption that every task is a valid trial. The company proposes …
A developer built Khidi, an API gateway that makes cheap open-weight AI models as easy to use as expensive proprietary ones by handling the plumbing of inference sourcing, key management, and billing.…
OpenAI published a research note on July 8, 2026, detailing reliability concerns in SWE-Bench Pro, emphasizing that coding benchmark scores are only meaningful when task design, grading, and potential…
AI Gateway has become the de facto routing layer for agentic workloads, with Meta's Muse Spark 1.1 and OpenAI's GPT-5.6 family now available through it. Muse Spark 1.1 offers 1M-token context and nati…
SpaceXAI released Grok 4.5 on July 8, claiming it matches Claude Opus 4.8 in capability while using far fewer tokens. Independent testing by Artificial Analysis confirmed Grok 4.5 uses over 60% fewer …
OpenAI retracted its recommendation for Scale AI's SWE-Bench Pro coding benchmark after an audit found roughly 30% of its 731 tasks were broken due to overly strict tests, underspecified prompts, low-…
OpenAI retracted its recommendation of SWE-Bench Pro after an audit found roughly 30% of its 731 tasks are broken. The audit, combining AI investigator agents and human reviewers, identified 200–249 b…
OpenAI released the GPT-5.6 family of models in three sizes—Luna, Terra, and Sol—claiming superior long-running agentic performance over Anthropic's Claude Fable 5 on the Agents' Last Exam benchmark, …
SpaceX Corp released Grok 4.5, a 1.5 trillion parameter AI model trained on tens of thousands of GB300 GPUs and built in collaboration with the Cursor IDE. The model uses approximately 16,000 tokens p…
OpenAI reviewed the SWE-Bench Pro AI coding test and found roughly 30% of its tasks are flawed, leading the company to withdraw its endorsement. The problems include overly strict or vague task requir…
OpenAI's audit of SWE-Bench Pro found that roughly 30% of the benchmark's tasks are broken, with issues including overly strict hidden tests, underspecified prompts, and low-coverage tests. The findin…
OpenAI announced it no longer recommends SWE-Bench Pro as a benchmark for evaluating coding models, citing concerns that the metric may not accurately reflect real-world software engineering performan…