A PlainEnglish article warns that AI benchmark scores such as MMLU, HumanEval, and HellaSwag can overstate production readiness when leaderboard numbers are treated as proof of model quality. The comm…
Atelier, a 30-second install tool that sits underneath Claude Code to reduce token waste, claims 30% savings on AI coding costs by providing better search, shorter file reads, compact command output, …
Anthropic released Claude Opus 4.7 on April 16, 2026, a targeted upgrade focused on making agentic coding loops and multi-step tool use production-ready. The model achieves 87.6% on SWE-bench Verified…
Z.ai's open-weight GLM 5.2 completed an agentic coding task in 17 minutes at $2.76, while Anthropic's Claude Fable 5 finished in 9 minutes at over $10, with comparable output quality. The open model c…
Anthropic released Claude Opus 4.7 on April 16, 2026, as its most capable public model, featuring a 13% improvement on coding benchmarks, a new xhigh effort level for agentic tasks, and high-resolutio…
Claude Sonnet 5 launched June 30 as the default model across Claude Code, Free, and Pro plans, but developers face three breaking API changes: sampling parameters (temperature, top_p, top_k) now retur…
New AI coding benchmarks, including Snorkel AI's Senior SWE-Bench and Scale AI's SWE-Bench Pro, are replacing older evaluations like HumanEval and original SWE-bench to test senior-level engineering s…
Public AI benchmarks like SWE-bench measure performance on popular open-source repositories but fail to predict how models will perform on proprietary codebases, team-specific conventions, and real-wo…
Google's Gemini 2.5 Pro with Deep Think reasoning mode topped coding and reasoning benchmarks this week, scoring 82.4% on GPQA Diamond and 94.1% on HumanEval+, but the mode multiplies token costs by r…
A developer argues that retrieval-augmented generation (RAG) for codebases improves context but not verifiability, citing that 30% of failed SWE-agent runs still claimed success. They introduce 'truth…
GitHub announced two harness-level improvements to Copilot agentic sessions—prompt caching achieving 94% cache hit rates and deferred tool loading—plus a new Auto model selection feature using its HyD…
Claude Code can improve other AI agents without training data in four of seven tested applications, performing as well as with data. Data helps only where Claude Code's prior knowledge of the task run…
A developer released state-harness, an open-source Python library that uses Lyapunov stability theory to detect and classify failure patterns in multi-turn LLM agents without extra LLM calls. The tool…
Institutional backlash against AI is mounting across public sector, media, and education, with New York City council members urging a classroom AI pause and a German newsroom caught using AI to write …
A technical guide argues that developers should run both Claude Code and OpenAI Codex as a single pipeline rather than choosing one, based on two months of testing on large codebases. Benchmarks show …
DeepSWE v1.1 updates the benchmark for long-horizon engineering tasks with isolated verification and structured test reports, making results more reproducible and harder to game. Pass rates remain clo…
RTK, a tool that compresses terminal output for LLM agents, claims to cut token usage by 60-90% but faces skepticism due to misleading savings metrics, silent failure risks, lack of accuracy benchmark…
Clawmark, an open-source Rust CLI tool, enables A/B testing of CLAUDE.md files by evaluating two variants against five SWE-bench Lite tasks using Claude and Docker. The tool generates a comparison rep…
Synaxi-predict, a new tool from Synaxi, predicts the token cost, turn count, and pass rate of a Claude Code task before execution, enabling users to select the optimal model and avoid wasted tokens. T…
Anthropic released Claude Fable 5 on June 9, but the U.S. Commerce Department issued an export-control directive barring foreign nationals from accessing it, forcing Anthropic to take both Fable 5 and…