{"slug": "introducing-physical-engineerings-last-exam-for-agents", "title": "Introducing Physical-Engineering’s Last Exam for agents", "summary": "Google's AI & Infrastructure team, in collaboration with Mercor, released Physical-Engineering's Last Exam (PLE), a benchmark of 85 long-horizon tasks across mechanical (30), electrical (27), civil (18) and structural (10) engineering, with 29 public tasks, grading tools and an open leaderboard at ple-bench.com. Testing 14 frontier models across seven agent harnesses (Antigravity, Claude Code, Codex, Hermes, OpenCode, OpenHands and Pi), Google reported that all frontier models score below 50% on PLE, which requires agents to work through an average of 61 files per task (5,170 total) and pass every hard safety check before earning points. The benchmark was built from 44 expert-authored tasks and 41 co-authored tasks assembled by an agent pipeline and validated by licensed practicing engineers.", "body_md": "Authors: Burak Gokturk, VP Research & Development; Rujun Han, Research Scientist; George Lee, Product Manager.\n\nToday, the [Google AI & Infrastructure](http://aiinfra.google) team is announcing **Physical-Engineering's Last Exam (PLE)**, a new benchmark and open competition, developed in collaboration with Mercor, and built to test a frontier model’s ability to complete tasks in mechanical, electrical, civil and structural engineering. \n\nTo help the research community measure and improve how agents handle physical engineering work, we are releasing 29 public tasks, their grading tools, and an open competition leaderboard at [ple-bench.com](http://ple-bench.com).\n\nFrontier labs and the broader research community use model examinations to test AI’s progress in completing tasks in a specific discipline. [Humanity's Last Exam](https://lastexam.ai/) measures a model’s general knowledge. [Agents' Last Exam](https://agents-last-exam.org/) measures a model’s ability to complete digital professional work. **Physical-Engineering's Last Exam (PLE)** measures a model’s ability to complete work that helps build the physical world. Physical engineers design skyscrapers, power grids, and computer chips we rely on, yet we could find no existing benchmark that tests whether frontier models can do that work end to end.\n\nFrontier language models now score above 90% on many academic benchmarks. However, those benchmarks mostly test textbook-style questions, single tool calls, or small, self-contained design problems. For AI developers and researchers building agents for physical  engineering, existing benchmarks do not show whether an agent can produce safe, code-compliant, and cost-effective designs under a licensed engineer's review and control. As the [demand for infrastructure development outpaces financing](https://infrastructurereportcard.org/wp-content/uploads/2025/03/Full-Report-2025-Natl-IRC-WEB.pdf?_gl=1*1timmgv*_up*MQ..*_ga*MzEyMjk2NDYxLjE3OTMwNDg0MTc.*_ga_C9EX6BGYX5*czE3OTMwNDg0MTYkbzEkZzAkdDE3OTMwNDg0MTYkajYwJGwwJGgw), there is a growing opportunity for agents to support engineers working to close this gap.   \n\nPLE closes this evaluation gap with 85 long-horizon tasks across four core disciplines: **mechanical** (30 tasks), **electrical** (27 tasks), **civil** (18 tasks), and **structural** (10 tasks) engineering.\n\nInstead of answering a single prompt, an agent works inside a containerized project folder — built on [Mercor’s Archipelago environment](https://github.com/Mercor-Intelligence/archipelago) — that mirrors what a practicing engineer receives on the job. Each task holds an average of 61 files (5,170 total files across the benchmark), including PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models, and document revisions. The agent must reason through these files, find the governing building codes and physical limits, run calculations, and generate production-ready deliverables that a licensed reviewer would review and approve.\n\nIn collaboration with experts from Mercor and licensed practicing engineers from industry, we built PLE using two methods:\n\nExpert-authored tasks (44 tasks): Practicing engineers created the project folders, prompts, and grading rubrics based on real professional engineering tasks.\n\nCo-authored tasks (41 tasks): Practicing engineers designed the initial task and an agent pipeline assembled the task's environment and grader from licensable engineering standards. Engineers then validated and increased the difficulty of every task assembled by the agent.\n\nWe tested 14 frontier models on PLE using seven publicly available agent harnesses (Antigravity, Claude Code, Codex, Hermes, OpenCode, OpenHands, and Pi). Because PLE evaluates both code compliance and cost efficiency, agents only earn points if their design passes every hard safety check first. Our baseline results show a wide gap between current models and real-world engineering requirements:\n\nHigh headroom: All frontier models score below 50% on our private test set, with the highest-performing model on its native harness reaching a mean score of 48.3%.\n\nHard safety gates: While the top-scoring agent configuration satisfied 91% of individual constraints on co-authored tasks, only 41.3% of its deliverables passed every hard safety and feasibility check required.\n\nModel vs. harness impact: Choice of model drives performance far more than the harness (up to a 50-point spread across models, versus a median 3-point spread across harnesses for the same model), though harnesses significantly affect run cost.\n\nEvaluating AI agents on realistic project files, strict safety rules, and material costs is a practical step toward building helpful tools for physical engineers. Here is how you can get started with PLE today:\n\nCheck out the full technical breakdown in our [PLE paper on arXiv](https://arxiv.org/pdf/2610.30537).\n\nDownload the 29 public tasks, grading scripts, and evaluation container from our [GitHub repository](https://github.com/google-research/ple).\n\nBring your own agent setup and submit it for evaluation on our private test set at [ple-bench.com](http://ple-bench.com)—with 13 Google-sponsored models running free on our servers and prize details coming soon.", "url": "https://wpnews.pro/news/introducing-physical-engineerings-last-exam-for-agents", "canonical_source": "https://cloud.google.com/blog/products/ai-machine-learning/introducing-physical-engineerings-last-exam-for-agents/", "published_at": "2026-11-03 08:00:00+00:00", "updated_at": "2026-11-03 17:22:01.529856+00:00", "lang": "en", "topics": ["ai-research", "ai-agents", "large-language-models", "artificial-intelligence", "ai-tools"], "entities": ["Google", "Mercor", "Physical-Engineering's Last Exam", "Burak Gokturk", "Rujun Han", "George Lee", "Archipelago", "ple-bench.com"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/introducing-physical-engineerings-last-exam-for-agents", "markdown": "https://wpnews.pro/news/introducing-physical-engineerings-last-exam-for-agents.md", "text": "https://wpnews.pro/news/introducing-physical-engineerings-last-exam-for-agents.txt", "jsonld": "https://wpnews.pro/news/introducing-physical-engineerings-last-exam-for-agents.jsonld"}}