{"slug": "metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in", "title": "METR reports AI capabilities scaling exponentially but reveals significant weakness in judgment tasks", "summary": "AI evaluation nonprofit METR's updated capabilities scaling report finds AI models continue improving along exponential trajectories on technical tasks but show a substantial capability gap in judgment-based work, according to Technology Review's coverage of the findings. METR's results indicate models perform significantly better on narrow technical benchmarks than on broader reasoning requirements.", "body_md": "# METR reports AI capabilities scaling exponentially but reveals significant weakness in judgment tasks\n\nAccording to AI evaluation nonprofit METR, an updated capabilities scaling report shows AI models continue improving along exponential trajectories for technical tasks but reveal a substantial capability gap in judgment-based work. Technology Review reported METR's findings suggest models perform significantly better at narrow technical benchmarks than on broader reasoning requirements.\n\n## Topics\n\n## Sources\n\nThis intelligence is sourced automatically from public sources across the web and synthesised by the Prefactor AI pipeline. Stories are reviewed before publication.", "url": "https://wpnews.pro/news/metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in", "canonical_source": "https://www.getreadyforagents.com/news/metr-capabilities-scaling-judgment-gap/", "published_at": "2026-11-20 19:07:08+00:00", "updated_at": "2026-11-20 19:14:32.069763+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-research", "ai-safety", "large-language-models"], "entities": ["METR", "Technology Review", "Prefactor AI"], "also_reported_by": [], "takeaway": "METR's scaling report shows models keep improving exponentially on narrow technical benchmarks but lag substantially on judgment-based tasks, so evaluations should test broader reasoning rather than technical benchmarks alone.", "recommendation": {"advice": "Pair narrow technical benchmarks with judgment-based reasoning evals when assessing models; METR found a substantial gap between the two.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in", "markdown": "https://wpnews.pro/news/metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in.md", "text": "https://wpnews.pro/news/metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in.txt", "jsonld": "https://wpnews.pro/news/metr-reports-ai-capabilities-scaling-exponentially-but-reveals-significant-in.jsonld"}}