Public AI benchmarks should be used as capability screens to build a shortlist rather than as deployment decisions, according to a practical evaluation framework published by deepsense.ai. The framewo…
Blitzy Inc. raised $200 million at a $1.4 billion valuation in May to run thousands of coding agents in parallel, and the company's director of engineering, Neeraj Deshmukh, said the real obstacle for…
The UK AI Security Institute (AISI) is publishing verified evaluation results through the EvalEval Coalition's Evaluation Cards platform, covering five benchmarks — HealthBench, FrontierMath, Humanity…
VS Code 1.135, released August 26, introduces Rubber Duck, a secondary AI agent that critiques the primary coding agent's plans, implementation, and tests before commits, using a model from a differen…
Microsoft released VS Code 1.135 on August 26, introducing the open, MIT-licensed Agent Host Protocol (AHP) that turns the editor into a universal agent runtime, external session continuity for pickin…
Researchers proposed AutoSaddler, an automatic harness optimization framework that improves LLM agent reliability on long-horizon tasks by treating harness updates as code patches derived from failure…
VS Code 1.135, released August 26, introduces the Agent Host Protocol (AHP), external agent sessions, and a new 'Rubber Duck' feature that pairs a second AI model to review the primary agent's work. T…
VS Code 1.135, released August 26, introduces the Agent Host Protocol (AHP), an open, MIT-licensed specification that decouples AI agent sessions from the editor window, allowing sessions to persist i…
SWE-bench remains the most widely used open-source benchmark for AI coding agents, with 2,294 tasks from 12 Python repositories, but newer benchmarks like Terminal-Bench, SWE-Bench Pro, and Senior SWE…
Anthropic's Claude Sonnet 5, priced at $2/$10 per million tokens during an introductory period (standard $3/$15 after August 2026), matches or nearly matches the flagship Claude Opus 4.8 on knowledge …
XAI released Grok 4.6 on August 7, a language model built on the same 1.5 trillion-parameter V9 foundation as Grok 4.5 but with improved post-training, including better supervised fine-tuning and rein…
Researchers introduced SWE-Touch, a benchmark evaluating coding agents' ability to repair software after users directly edit code, finding that most models' performance drops significantly when user e…
Alibaba released Qwen 3.8 Max, a 2.4 trillion parameter open-weight AI model with 95 billion active parameters, set to become the largest open-weight model once weights are open-sourced about a week a…
Medley, a harness system that orchestrates model workers with planning, routing, and acceptance stages, achieved the highest reported results on four public benchmarks: Terminal-Bench 2.1 (93.25% task…
Vercel's AI Gateway now offers Laguna S 2.1, an open-weight Mixture-of-Experts model from Poolside that supports up to 1M tokens and agentic coding tasks. The model achieves 70.2% on Terminal-Bench 2.…
OpenAI published an audit of SWE-Bench Pro on July 8, 2026, estimating that roughly 30% of its tasks are broken, which undermines the assumption that every task is a valid trial. The company proposes …
A developer built Khidi, an API gateway that makes cheap open-weight AI models as easy to use as expensive proprietary ones by handling the plumbing of inference sourcing, key management, and billing.…
OpenAI published a research note on July 8, 2026, detailing reliability concerns in SWE-Bench Pro, emphasizing that coding benchmark scores are only meaningful when task design, grading, and potential…
AI Gateway has become the de facto routing layer for agentic workloads, with Meta's Muse Spark 1.1 and OpenAI's GPT-5.6 family now available through it. Muse Spark 1.1 offers 1M-token context and nati…
SpaceXAI released Grok 4.5 on July 8, claiming it matches Claude Opus 4.8 in capability while using far fewer tokens. Independent testing by Artificial Analysis confirmed Grok 4.5 uses over 60% fewer …