{"slug": "berkeley-benchmark-finds-agents-fail-most-job-tasks", "title": "Berkeley Benchmark Finds Agents Fail Most Job Tasks", "summary": "UC Berkeley RDI's Agents' Last Exam (ALE) benchmark, released in June 2026, found that every tested frontier AI agent scored 0% on its hardest tier of long-horizon professional tasks, while PYMNTS reported on July 24 that Codex running GPT-5.5 completed 26.2% of assignments in a broader configuration. The benchmark covers 55 occupational subdomains with over 1,500 expert-sourced tasks and uses outcome-based grading to test economically valuable work.", "body_md": "# Berkeley Benchmark Finds Agents Fail Most Job Tasks\n\nUC Berkeley RDI released Agents’ Last Exam in June 2026 to test AI agents on long-horizon professional work with verifiable outcomes. PYMNTS reported on July 24 that Codex running GPT-5.5 completed 26.2% of the broader assignment set, while Berkeley’s release reported a 0% pass rate for every tested frontier agent on its hardest tier; the figures reflect different configurations.\n\nUC Berkeley RDI released **Agents’ Last Exam (ALE)** in June 2026 to test AI agents on long-horizon, economically valuable professional work with verifiable outcomes. The benchmark covers 55 occupational subdomains and uses tasks derived from work completed by domain experts.\n\nPYMNTS reported on July 24 that the best system in the broader results it reviewed, Codex running GPT-5.5, completed **26.2%** of assignments. Berkeley RDI’s release separately says every frontier agent it tested scored **0%** on ALE’s hardest tier. Those figures should not be treated as the same measurement.\n\n### What ALE evaluates\n\nBerkeley describes ALE as a response to benchmarks that measure narrow or short-horizon skills. Its tasks span fields including science, engineering, medicine, law, finance, and education, and can require graphical interfaces, command-line tools, or both.\n\nThe project’s public materials report slightly different task and contributor totals as the benchmark evolved. Berkeley’s release describes more than 1,500 expert-sourced tasks and more than 300 experts from over 100 institutions. The paper and PYMNTS coverage describe 1,490 tasks collected from more than 250 professionals. The consistent design goal is outcome-based grading of realistic deliverables rather than subjective impressions of an agent’s process.\n\nThe official GitHub repository contains the open evaluation framework, about 150 public tasks, sandbox provisioning, agent integrations, and deterministic grading tools. Hidden references are staged after an agent finishes so the expected deliverable is not exposed during the run.\n\n### Why the headline scores differ\n\nPYMNTS reports a 26.2% overall completion rate for Codex with GPT-5.5 and an average pass rate of 2.6% across systems on the hardest assignments, with Codex reaching 8.6% on that subset. Berkeley’s June release says every frontier agent in the configuration it highlights, including Fable 5, scored 0% on its hardest tier.\n\nThe apparent conflict is a warning against comparing agent scores without the model version, agent harness, task subset, benchmark revision, and evaluation run. A model can be paired with different planning loops, tools, memory, and execution environments, and those system-level choices affect the result.\n\n### Practical implications\n\nALE’s strongest contribution is not a single leaderboard number. It tests whether an agent leaves behind a correct, complete artifact after sustained work. Berkeley says a common failure mode is premature success: the agent declares a task finished even when required files, counts, fields, or constraints are wrong.\n\nFor teams evaluating autonomous workflows, that is a production concern. Demonstrations can hide compounding errors in state tracking, tool use, verification, and recovery. Task-specific deterministic grading, held-out references, and explicit checks for complete deliverables provide a more useful readiness signal than a polished transcript or a broad chatbot benchmark alone.\n\n## Key Points\n\n- 1ALE evaluates long-horizon professional deliverables across 55 occupational subdomains using verifiable, outcome-based grading.\n- 2PYMNTS reported a 26.2% best overall result, while Berkeley reported 0% for every tested agent on the hardest tier in the configuration it highlighted.\n- 3Teams should compare agent scores only with the model, harness, task subset, benchmark revision, and grading setup specified.\n\n## Scoring Rationale\n\nALE is a broad, reproducible evaluation of agent performance on economically valuable tasks and is directly relevant to teams assessing autonomous-workflow claims. Its low completion rates and transparent configuration caveat provide useful evidence on reliability limits, though the benchmark is new and its public materials reflect multiple revisions and harnesses.\n\n## Sources\n\nPrimary source and supporting public references used for this report.\n\nPractice interview problems based on real data\n\n1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.\n\n[Try 250 free problems](/problems)", "url": "https://wpnews.pro/news/berkeley-benchmark-finds-agents-fail-most-job-tasks", "canonical_source": "https://letsdatascience.com/news/berkeley-benchmark-finds-agents-fail-most-job-tasks-40c5913c", "published_at": "2026-07-24 17:16:37+00:00", "updated_at": "2026-07-24 19:26:55.312451+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-research"], "entities": ["UC Berkeley RDI", "Agents' Last Exam", "Codex", "GPT-5.5", "PYMNTS", "Fable 5"], "alternates": {"html": "https://wpnews.pro/news/berkeley-benchmark-finds-agents-fail-most-job-tasks", "markdown": "https://wpnews.pro/news/berkeley-benchmark-finds-agents-fail-most-job-tasks.md", "text": "https://wpnews.pro/news/berkeley-benchmark-finds-agents-fail-most-job-tasks.txt", "jsonld": "https://wpnews.pro/news/berkeley-benchmark-finds-agents-fail-most-job-tasks.jsonld"}}