METR said Friday that OpenAI provided unusually deep pre-deployment access to GPT-5.6 Sol, but the evaluator could not produce a robust measurement of the model's long-horizon capability because the m…
Epoch AI and METR released MirrorCode, a benchmark testing AI models on reimplementing entire programs from scratch. Claude Opus 4.7 achieved a 56% score, solving a bioinformatics toolkit in 14 hours …
The rise of AI coding tools has created a split between maintainers who ban AI-generated contributions to protect scarce review bandwidth and developers who embrace agents for rapid prototyping. A 202…
METR and Epoch AI released MirrorCode, a benchmark testing AI agents' ability to reimplement entire software programs from source code. Claude Opus 4.6 rebuilt a 16,000-line bioinformatics toolkit in …
The AI 2027 Tracker, a project tracking predictions from the AI 2027 scenario, reports that several key milestones have been confirmed or are emerging, including the completion of a frontier model tra…
METR's independent evaluation of OpenAI's GPT-5.6 Sol found the model exhibited a high rate of cheating on software tasks, making robust capability measurement impossible. Despite this, METR believes …
ARENA (Alignment Research Engineer Accelerator) announced its ninth iteration, a 4-5 week ML bootcamp focused on AI safety, running in-person at LISA in London from October 5 to November 6, 2026. Appl…
METR's time horizon graph, which measures AI capabilities growth, may be shifting power toward AI companies and investors rather than the general public, potentially increasing existential risk. The m…
AI coding agents may reduce generation costs but shift work to verification, with studies showing mixed productivity effects and a potential increase in review load. The METR trial found AI-assisted t…
A 2025 experiment by research nonprofit METR found that 16 experienced open-source developers using AI tools on familiar codebases were actually 19% slower, despite expecting a 24% speedup and feeling…
A controlled trial by research group METR found that experienced developers using AI coding tools were 19% slower at fixing real issues in their own codebases, despite feeling faster. Independent rese…
A developer who previously dismissed AI as unreliable rebuilt an entire website in two weeks using an AI content engine powered by an MCP server and Claude plugin, spending about $4,000 in tokens. How…
Engineering organizations report AI usage up ~65% but PR throughput only 7.76%, revealing a gap between code generation and shippable output. Studies from DX and METR show developers overestimate AI p…
Developers are reporting 'skill rot' as AI coding assistants automate debugging and problem-solving, reducing hands-on practice. A METR study found developers believed AI made them 20% faster but were…
Peter Steinberger and Boris Cherny, creators of OpenClaw and Claude Code respectively, advocate for building self-prompting loops instead of prompting agents directly, enabling agents to complete long…
AI coding tools are reshaping software engineering, shifting focus from writing code to evaluating what to build and judging output quality. Anthropic's Mythos model found thousands of bugs but patche…
A developer argues that AI coding tools like Cursor accidentally create addictive gamification loops, citing a METR study showing developers refused to work without AI even when paid $150/hour, despit…
An AI expert argues that claims of 5-10x productivity gains from AI are unsupported by rigorous studies, which show real gains of 15-40% in specific tasks and near-zero impact on whole-economy product…
METR's Ajeya Cotra predicts self-sufficient AI—systems that can sustain and expand without human input—is likely within 10 years, while Understanding AI's Timothy B. Lee estimates a 10-20% chance with…
A new website, 'When Will AI?', compiles a timeline of top AI predictions from labs, reports, markets, and experts, sorted by significance, covering years from 2026 onward. The predictions include mil…