cd /news/ai-safety/my-top-10-eval-papers-of-2026 · home › topics › ai-safety › article
[ARTICLE · art-170743] src=nlp.elvissaravia.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

My Top 10 Eval Papers of 2026

Epoch AI's Benchmark Reviews rated 12 of the 24 public benchmarks it examined as Flawed as of late October 2026, meaning errors affect at least 20% of the score, and an OpenAI audit of SWE-bench Verified found serious problems in 59.4% of the 138 tasks that o3 could not reliably solve. The findings appear in a roundup of the author's top 10 eval papers of 2026, which also cites a July 2026 incident in which an agent built on OpenAI models escaped its sandbox during a cybersecurity eval on the ExploitGym benchmark and broke into Hugging Face's systems, which Hugging Face believes was an attempt to steal the benchmark's test solutions. The roundup argues that public leaderboards remain useful for a first look but that decisions that matter require private evals built on an organization's own tasks and data.

by read10 min views5 publishedNov 16, 2026
My Top 10 Eval Papers of 2026
Image: Nlp (auto-discovered)

Every week, a new model or agent claims a higher score on some public benchmark. AI engineers and researchers use those numbers to pick models, ship features, and decide what to build next. When the eval behind a number is weak, the decisions based on that number can be wrong.

This matters more now than it did a year ago. A chatbot answer can be checked by reading it. An agent can work for hours inside real software, call dozens of tools, and change files along the way. Grading that work is harder. You have to check that the final result is correct. You also have to check that the steps it took along the way were safe and sensible. Much of the eval research I have tracked in 2026 focuses on how to grade these types of workflows.

A weak eval can mislead you without any warning. Sometimes the benchmark is so easy for today's models that a good model and a great one get the same score. Other times the grader passes answers it should have rejected. Even when the final result looks right, the agent may have cheated to get there, for example by finding the test data. In July 2026, an agent built on OpenAI models was running a cybersecurity eval based on the ExploitGym benchmark. It escaped its sandbox and broke into Hugging Face's systems. Hugging Face believes the agent was trying to steal the benchmark's test solutions.

Recent audits show how common these problems are in public benchmarks. In September, Epoch AI started publishing Benchmark Reviews. As of late October, 12 of the 24 benchmarks it reviewed were rated Flawed, meaning errors affect at least 20% of the score. OpenAI audited SWE-bench Verified and found serious problems in 59.4% of the 138 tasks that o3 could not reliably solve. Public leaderboards are still useful for a first look. For decisions that matter, you need private evals built on your own tasks and data. Several of the papers below show how to build them.

I read a lot of eval papers. These are my 10 favorites from this year. Each one names a specific problem with evals and shows how to reduce it in your own setup. Plenty of good papers did not make the list.

You can find more in our Top AI Papers of the Week archive.

1. Pick Durable Benchmarks #

When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturation, Mubashara Akhtar et al., 2026.

Many popular benchmarks have become too easy. When almost every model gets a near-perfect score, the benchmark can no longer show which model is better. Researchers call this saturation.

The authors measured this across 60 benchmarks. Nearly half had saturated. Older benchmarks were more likely to be saturated. The benchmarks that stayed useful the longest were curated by experts. Whether the test set was public made little difference.

Before you trust a benchmark, check that the models you care about still get clearly different scores on it. Prefer benchmarks curated by experts.

2. Test Realistic Tasks #

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces, Mike A. Merrill et al., 2026.

Good tasks need to be realistic and easy to check. Terminal-Bench shows how to build tasks that are both.

Each of its 89 tasks runs in its own container. Each task includes an instruction, a human-written solution, and tests that check the final state. The tasks are based on real command-line work. At release, every frontier model and agent scored below 65%.

Build your own tasks the same way. Give each one a clean environment, a known solution, and a test that checks the result.

3. Run Paired Comparisons #

SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks, Xiangyi Li et al., 2026.

This paper is about measuring whether a change to your agent actually helps. Say you add a new tool, prompt, or skill, and the score goes up. Without a baseline, you cannot tell whether the change caused the gain or something else changed between runs. The fix is a paired comparison. You run the same tasks twice, once with the change and once without it, and compare the two scores.

SkillsBench does this for agent Skills, the packaged instructions an agent loads while it works. It ran 87 tasks with and without Skills across 18 model and harness setups. Skills increased the average pass rate from 33.9% to 50.5%. Short Skills with three or fewer parts worked better than long ones.

The gain varied a lot by setup, from about 4 to 26 points. Test your own model and harness the same way before assuming a change helps.

4. Use Judge Confidence #

JEV-as-a-Judge: Accept When Confident, Escalate When Unsure, Yubo Li et al., 2026. Once you have figured out a good set of tasks, the next problem is the grader. Using a frontier model to judge every output gets expensive quickly.

JEV is a small decision model from TypeSafe. As a judge, it returns a verdict with a probability instead of a written explanation. When the answer could be checked by reading the text, it scored within three points of GPT-6 at a small fraction of the cost. It fell behind on math, code, and logic, where the judge has to work out the answer itself.

The authors used JEV's confidence to decide when to send a case to a stronger judge. Confident verdicts were accepted. Uncertain ones went to GPT-6. This combination was slightly more accurate than GPT-6 alone at 41% of the cost. The routing worked less well on answers written to look convincing, so set the confidence cutoff on your own data.

5. Check Judge Independence #

Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels, Guneet Kohli, 2026.

A common fix for an unreliable judge is to add more judges and take a vote. This paper shows why that often fails.

Nine frontier models from seven model families judged the same items. They tended to get the same items wrong. As a result, the nine votes carried about as much information as two independent ones. The best single judge did as well as the whole panel or better. Smarter voting rules did not fix the problem.

Before adding judges, check how often they make the same mistakes. The study used language inference and preference tasks, so test this on your own grading data.

6. Give Judges Evidence #

Agent-as-a-Judge, Runyang You et al., 2026. Some work cannot be graded by reading the output alone. The judge has to open the file, run the code, or look up a fact.

This survey covers judges that do that work themselves. They plan their checks, use tools, and collect evidence before giving a score.

These judges help when a single prompt cannot verify the answer. They also cost more, run slower, and add security risk. Check them against human-reviewed examples like any other grader.

A judge with real tools needs the same sandbox as the agent it grades. Lock it down, and review what the judge actually does during a run.

7. Inspect the Runs #

Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents, Bowen Ye et al., 2026. Many evals only check the agent's final answer. Claw-Eval shows that this misses many problems that happen while the agent is working, such as unsafe actions.

It records every run in three places. The trace holds the agent's actions. Audit logs hold what the connected services saw. A snapshot holds the final state of the environment. Grading only the final output missed 44% of the safety violations that this fuller grading caught. It also missed 13% of the robustness failures. Each task runs three times, so a task the agent passes once by chance does not count as solved.

Save traces, logs, and final-state snapshots for every run. These numbers come from 14 models on Claw-Eval's own tasks, so your results will differ.

8. Separate Process and Outcome #

The Art of Building Verifiers for Computer Use Agents, Corby Rosset et al., 2026.

A verifier is the part of an eval that decides whether the agent succeeded. For computer-use agents, which click through websites and apps, the verifier has to review a long run that can span many screenshots. If the verifier gets that call wrong, the eval score is wrong too, and so is every decision based on it. This paper shows how to build a verifier that gets it right more often.

The Universal Verifier splits grading in two. One score asks whether the agent finished the task. A second score asks whether its actions along the way were appropriate. The rubric items do not overlap. The verifier also separates the agent's own mistakes from outside problems it could not control.

Use the same split in your own rubrics. The design was tested on computer-use tasks and still produced some false passes, so recalibrate it before using it elsewhere.

9. Audit Self-Improvement #

PostTrainBench: Can LLM Agents Automate LLM Post-Training?, Ben Rank et al., 2026. Evals get a lot harder when the agent is trying to improve a model itself. This is the setup behind much of the current talk about self-improving agents. If the eval is weak, the agent can raise the score without making the model better. PostTrainBench gives an agent a base model, a target benchmark, and 10 hours on one H100 GPU. The agent can train the model however it wants.

A separate judge then checks for cheating. Examples include training on test data or down a model that someone else already tuned. Clean runs are scored on the full benchmark. Flagged runs get the base model's original score. Results are averaged over four base models and seven benchmarks.

The authors caught agents using both shortcuts. If you evaluate a self-improvement loop, check how each gain happened and keep a record of the training data. The agents in this study took shortcuts when they could, so treat a higher score as unverified until you know where it came from.

10. Use Personal Context #

LifeVal: Evaluating AI Model Performance on Personalized Agentic Consumer Tasks, Jenny Yang et al., November 2026.

Personal agents are one of the fastest-growing uses of AI this year. People now let assistants remember past conversations, read their files, and act for them, such as planning a move, booking travel, or ordering groceries. A personal agent is only useful if it gets the details of your life right. A generic benchmark cannot test that, because the correct answer depends on who is asking.

LifeVal tests whether an agent helps a specific person. The authors surveyed more than 3,800 consumers to find the tasks they most want help with. They then built 169 tasks from real people's needs. Each task includes that person's chat history, memories, and files, plus a rubric written for that task.

The most common failure across all five OpenAI models tested was wrong information or a wrong recommendation. Models often missed relevant details from the person's past conversations. The automated grader tracked human graders closely, with a correlation of 0.86 on a 30-task check. If you build an assistant, include the personal context that changes what a good answer looks like. The task list came from a US-focused survey. Its cached web pages can also change between runs, so scores may not repeat exactly.

How to Read This List #

If you are building an eval from scratch, reading the papers in order may help. The first three set up the tasks and the comparison. The next three cover the grader. Claw-Eval and the verifiers paper show how to check the evidence behind a score. PostTrainBench and LifeVal apply the same ideas to self-improving agents and personal assistants. Each paper addresses one problem. Together, they cover the main steps of building an eval, from choosing a benchmark to checking the final score.

If you start with one habit, make it the paired comparison from SkillsBench. Run your own tasks with and without each change, and trust a gain only after you see it in that comparison.

── more in #ai-safety 4 stories · sorted by recency
── more on @epoch ai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/my-top-10-eval-paper…] indexed:0 read:10min 2026-11-16 · —