Four papers published today attack the same assumption from different directions: that a model's assessment of an output, its own or another model's, is reliable enough to label data or gate a loop. Function-level execution feedback compared judge annotations against actually running the code and found the judge marking working functions as failures, corrupting the preference data trained on them, while a separate study shows multi-dimensional judge rubrics leak across dimensions rather than scoring independently. The termination paper carries the point into agent design, replacing a model's self-declared doneness with a replayable evidence certificate it must produce before COMPLETE. If you label, score or gate with a judge, the cheap correction is not a better rubric, it is an executable check the judge cannot talk its way past. Read: Function-level execution feedback compared judge annotations against actually running the code, and found the judge marking working functions as failures, which poisons the preference data trained on them. A separate paper shows multi-dimensional judge rubrics are not independent, with one dimension's score leaking into another, and offers step-wise chain-of-thought pruning that reduces the leakage across models and tasks. A third reports near-zero correlation between automatic creativity metrics and human judgment, plus a systematic pull toward AI-styled text. Where the artifact can be run, run it. Read: Self-judged completion breaks under exactly the conditions agent loops produce, lost tool responses and partial commits, where the model reports done over a half-applied change. The proposal is a replayable evidence certificate the agent must carry before it may emit COMPLETE, which is a gate you can bolt onto an existing loop without retraining anything. Read: Qwen3.8-Flash-Next ships open weights for a multimodal MoE with roughly 6B active parameters out of 125B, priced near $0.16 and $0.47 per million tokens, which the lab frames as an early look at Qwen4. SemiAnalysis itemizes what is inside: a 51-billion-parameter n-gram embedding table cheap enough to push onto slower memory tiers, gated residual connections, and a lightning indexer selecting context at micro-block granularity. The architecture can be read rather than inferred, which is the part worth your afternoon. Read: ToolRobustBench perturbs interface, intent, output and runtime as separate stages, and reports tool-calling accuracy degrading sharply and unevenly, with stage attribution naming which part of the harness gave way. ESQ-Bench finds the enterprise version of the same gap, NL2SQL accuracy above 89% on academic sets falling to between 57% and 80% on tiered Oracle schemas, and its silent-divergence metric catches the worst case: queries that execute cleanly and return the wrong rows. Try: Codex rust-v0.150.0 suppresses AGENTS.md in untrusted projects, so a cloned repository can no longer hand instructions to the agent reading it. Cline v4.1.16 redacts credentials embedded in git remote URLs before workspace information reaches the model. Two unrelated tools shipped the same premise the same day: the working directory is model input, and input from strangers gets sanitized first. Read: Anthropic opened real Claude and Claude Code conversation data to independent researchers, and the first result reports that over half of sampled conversations involved consequential, hard-to-undo work. That figure is the case for the gates the papers above describe, since it measures how much everyday agent use already carries irreversible side effects. A METR productivity study is queued behind it. Read: Cowork's desktop app gains an Anthropic-controlled browser in a side panel that navigates and fills forms inside a session held separate from the user's own profile and logins. The separation is the whole feature, since agentic browsing on a live cookie jar is the failure it avoids. Rolling out to paid plans over the next week, with Claude in Chrome now generally available.
Four papers land today against LLM-as-judge, and for executable checks
Four papers published today challenge the reliability of LLM-as-judge, showing that judge annotations can mark working code as failures, leak across rubric dimensions, and misalign with human creativity judgments, while a separate proposal replaces self-declared completion with replayable evidence certificates for agents. The papers, including ToolRobustBench and ESQ-Bench, also report sharp accuracy drops in tool-calling and NL2SQL tasks under real-world perturbations, and Anthropic's opened conversation data reveals over half of sampled Claude interactions involved consequential, hard-to-undo work.
Run your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.