cd /news/ai-safety/evals-are-moving-from-what-models-sa… · home › topics › ai-safety › article
[ARTICLE · art-172515] src=dev.to ↗ pub= topic=ai-safety verified=true sentiment=· neutral

Evals Are Moving From What Models Say to What Agents Do

A weekly LLM evaluation and observability digest highlights a shift in evaluation from judging what models say to judging what agents do, citing a ViKernel Scanner test in which 20 open models given a shell all failed action-level safety checks. It also rounds up arXiv work on cheaper judging, including ExecJudges, which uses one LLM call per topic to write an executable program that grades documents at zero marginal inference cost, and papers on judge reliability, Goodhart traps and risk-certified guardrails.

recommendation

Replace per-document LLM judging in CI with a one-call-per-topic executable program judge, and validate it against human labels before trusting its agreement rate.

more in playbook: Evals and testing for LLM apps
by read5 min views5 publishedNov 19, 2026

This digest covers what changed in LLM evaluation and observability between 2026-11-12 and 2026-11-19. It's a quieter week on the tooling side, but arXiv delivered a strong cluster of papers on judge reliability, judge cost, and what a "judge" even needs to look at anymore.

#

🔥 Highlights

We Handed 20 Open Models a Shell. 20 of Them Failed: Introducing ViKernel Scanner — 20 open models, all failed action-level safety.

Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling — one LLM call, then judge for free. XYBench: Helping with the problem behind the question — new benchmark for catching misread user intent.

[Agent-as-a-Judge](https://arxiv.org/abs/2611.18067) — judges need to gather evidence, not guess.
[GPT‑6.2 Astra and GPT‑6.2 Sol](https://simonwillison.net/2026/Nov/17/gpt62-astra-and-sol/) — a real-world vibe-check and cost comparison.

#

arXiv cs.CL

Can Programs Replace LLM Judges? Executable Program Judges for Scalable and Efficient Relevance Labeling (2026-11-18) — Instead of calling an LLM to judge every document, ExecJudges spends one LLM call per topic to write an executable program (lexical matching, standard NLP stats) that then does all the grading at zero marginal inference cost. Across 7 TREC collections it matched LLM-judge agreement with NIST human assessors at $0.17–$0.49 per topic total, with no recurring cost — a direct way to get LLM-as-judge quality without paying inference on every eval run in CI. #

FutureBench: A Renewable Framework for Evaluating Knowledge-Context Conflict in LLMs (2026-11-17) — A renewable benchmark (timestamped questions, 3,629 of them, 6 models tested) measuring how models resolve conflicts between their own knowledge and retrieved context, directly relevant to RAG pipeline testing. It shows a "verify the passage" prompt increases adoption of false retrieved answers, while a "beware misleading passages, trust your own knowledge" prompt recovers most of the lost accuracy. #

Jev-as-a-Reward: Validating Probabilistic Safety Judges for Reinforcement Learning (2026-11-16) — Proposes a protocol for validating probabilistic safety judges before, during, and after RL training, separating "confidently approved harm" (never escalated to a stronger judge) from total harm. Even frontier LLM judges catch at most 1 in 10 human-labeled "confident misses" — a concrete warning for anyone trusting automated confidence gates in safety/RLHF pipelines. #

Risk-certified prompt optimization for LLM-as-a-judge guardrails (2026-11-15) — Frames the system prompt of an LLM-as-judge guardrail as a constrained optimization: maximize benign-content approval while keeping undetected-attack rate under a statistically certified, finite-sample threshold. Tested across 4 datasets (prompt injection, system prompt leakage, sensitive data) — useful for teams that need to formally justify a guardrail threshold instead of tuning it by feel. #

Grounding Language Models in Speech Does Not Fix "AI Flavor": Calibrated LLM Judges, Goodhart Traps, and a Preference-Tuning Fix (2026-11-15) — Builds a calibrated LLM-judge protocol for "AI flavor" in Chinese text and turns the judge's agreement labels into DPO preference pairs. The judge's bias toward shorter responses collapses the trained model's output length — a textbook Goodhart trap and a direct lesson for anyone using an LLM judge as a reward signal without an SFT anchor against reward hacking. #

Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems (2026-11-15, revised 2026-11-17) — A pre-deployment audit framework for LLM-based assessment systems, testing operational performance, compliance (hidden instructions in résumés), and consistency between a model's verdict and its own stated justification. It finds the LLM's verdict frequently doesn't match its own justification — relevant to any eval setup that uses "judge explanation" as a proxy for reliability, well beyond hiring. #

Agent-as-a-Judge (2026-11-13) — Argues plain LLM-as-a-Judge fails when a verdict needs evidence outside the provided material (unlogged environment state, claims requiring external checking), and proposes judges that actively gather that evidence before ruling. Maps the design space (multi-agent, tool use, memory, planning) and closes with cost, reliability, and governance tradeoffs — a useful map for deciding whether a more complex, agentic judge is worth it.

#

Hugging Face Blog

We Handed 20 Open Models a Shell. 20 of Them Failed: Introducing ViKernel Scanner (2026-11-18) — ViKernel Scanner gives a model 8 mock tools (files, mail, web, shell, payments) and 100 attack checks plus normal-work controls, then grades on attempted actions rather than words. All 20 tested open-weight models scored D or F, with a median model attempting a dangerous action in 40 of 100 checks; replaying the same findings through a weight-embedded firewall denied 897 of 897 dangerous calls before execution. Open source (Apache-2.0) — a concrete argument for action-level agent evals over output-text scoring. #

XYBench: Helping with the problem behind the question (2026-11-13) — A new benchmark (8,115 questions from Stack Overflow/Stack Exchange/WikiHow) measuring whether a model recognizes an "XY problem" — a request built on a wrong premise — and redirects to the real goal instead of just answering literally. It uses Claude Sonnet 4.5 as an automated judge, validated against human review on 500 responses (74–80% agreement), which is useful for catching a specific regression class pure-accuracy metrics miss, as long as you calibrate the judge against your own human review first.

#

Langfuse Blog

Nothing new and relevant since 2026-11-12 — the most recent post ("Designing the Langfuse AI Gateway") is dated 2026-11-11, just outside this digest's window.

#

Braintrust Blog

Nothing new and relevant since 2026-11-12 — the latest post ("Manage Braintrust with Terraform") is dated 2026-11-10.

#

Simon Willison

GPT‑6.2 Astra and GPT‑6.2 Sol (2026-11-17) — Simon's first-look post on OpenAI's two new models (Astra, frontier; Sol, efficient), run through his recurring informal benchmarks — the "pelican riding a bicycle" test at multiple effort levels, plus a heavier 3D-scene generation task — with results and per-run cost compared against Claude Fable 5.2 and Gemini 4 Carbon. It's a useful real-world example of lightweight "vibe check" + cost-per-run benchmarking methodology for teams deciding which model to route to in production.

#

Interconnects

Nothing new and relevant since 2026-11-12 — the only post in the window ("Open-source AI without Jensen", 2026-11-14) is market/policy commentary without an evaluation or observability angle.

#

Through-line

Three independent sources converge on the same idea this week: scoring what a model says is no longer enough. ViKernel Scanner and Agent-as-a-Judge both argue that judging needs to look at actions and gathered evidence, not just text. Meanwhile ExecJudges and the risk-certified guardrail paper tackle the other half of the same problem — once you're judging in production or CI, you need that judgment to be cheap and statistically defensible, not just plausible.

What's catching your eye in LLM evals this week? Drop a comment below.

── more in #ai-safety 4 stories · sorted by recency
── more on @vikernel scanner 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/evals-are-moving-fro…] indexed:0 read:5min 2026-11-19 · —