UC Berkeley RDI released Agents’ Last Exam in June 2026 to test AI agents on long-horizon professional work with verifiable outcomes. PYMNTS reported on July 24 that Codex running GPT-5.5 completed 26.2% of the broader assignment set, while Berkeley’s release reported a 0% pass rate for every tested frontier agent on its hardest tier; the figures reflect different configurations.
UC Berkeley RDI released Agents’ Last Exam (ALE) in June 2026 to test AI agents on long-horizon, economically valuable professional work with verifiable outcomes. The benchmark covers 55 occupational subdomains and uses tasks derived from work completed by domain experts.
PYMNTS reported on July 24 that the best system in the broader results it reviewed, Codex running GPT-5.5, completed 26.2% of assignments. Berkeley RDI’s release separately says every frontier agent it tested scored 0% on ALE’s hardest tier. Those figures should not be treated as the same measurement.
What ALE evaluates
Berkeley describes ALE as a response to benchmarks that measure narrow or short-horizon skills. Its tasks span fields including science, engineering, medicine, law, finance, and education, and can require graphical interfaces, command-line tools, or both.
The project’s public materials report slightly different task and contributor totals as the benchmark evolved. Berkeley’s release describes more than 1,500 expert-sourced tasks and more than 300 experts from over 100 institutions. The paper and PYMNTS coverage describe 1,490 tasks collected from more than 250 professionals. The consistent design goal is outcome-based grading of realistic deliverables rather than subjective impressions of an agent’s process.
The official GitHub repository contains the open evaluation framework, about 150 public tasks, sandbox provisioning, agent integrations, and deterministic grading tools. Hidden references are staged after an agent finishes so the expected deliverable is not exposed during the run.
Why the headline scores differ
PYMNTS reports a 26.2% overall completion rate for Codex with GPT-5.5 and an average pass rate of 2.6% across systems on the hardest assignments, with Codex reaching 8.6% on that subset. Berkeley’s June release says every frontier agent in the configuration it highlights, including Fable 5, scored 0% on its hardest tier.
The apparent conflict is a warning against comparing agent scores without the model version, agent harness, task subset, benchmark revision, and evaluation run. A model can be paired with different planning loops, tools, memory, and execution environments, and those system-level choices affect the result.
Practical implications
ALE’s strongest contribution is not a single leaderboard number. It tests whether an agent leaves behind a correct, complete artifact after sustained work. Berkeley says a common failure mode is premature success: the agent declares a task finished even when required files, counts, fields, or constraints are wrong.
For teams evaluating autonomous workflows, that is a production concern. Demonstrations can hide compounding errors in state tracking, tool use, verification, and recovery. Task-specific deterministic grading, held-out references, and explicit checks for complete deliverables provide a more useful readiness signal than a polished transcript or a broad chatbot benchmark alone.
Key Points #
- 1ALE evaluates long-horizon professional deliverables across 55 occupational subdomains using verifiable, outcome-based grading.
- 2PYMNTS reported a 26.2% best overall result, while Berkeley reported 0% for every tested agent on the hardest tier in the configuration it highlighted.
- 3Teams should compare agent scores only with the model, harness, task subset, benchmark revision, and grading setup specified.
Scoring Rationale #
ALE is a broad, reproducible evaluation of agent performance on economically valuable tasks and is directly relevant to teams assessing autonomous-workflow claims. Its low completion rates and transparent configuration caveat provide useful evidence on reliability limits, though the benchmark is new and its public materials reflect multiple revisions and harnesses.
Sources #
Primary source and supporting public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.