{"slug": "recoverability-and-verifier-strength-two-measured-preconditions-for-unattended", "title": "Recoverability and Verifier Strength: Two Measured Preconditions for Unattended Agent Self-Improvement", "summary": "A new arXiv paper (2610.27465v1) proposes two measurable preconditions — verifier strength and recoverability — for letting agent platforms ship self-improvement changes without human review. In tests on the platform's own approval gate and on the τ²-bench agent benchmark, the platform's LLM judge returned identical scores regardless of input in 9 of 37 settings without reporting an error, and a deployed change altered the outcome of 17 percent of tasks beyond normal run-to-run variation, a shift no undo repaired. The authors conclude a change should run unattended only when its old version is kept for rollback and its verifier's strength has been measured, a rule that permits prompt, tool and skill edits while keeping weight updates under human sign-off.", "body_md": "arXiv:2610.27465v1 Announce Type: new \nAbstract: Agent platforms are starting to improve their own agents: they read a deployed agent's execution traces, derive a change, have an automated verifier approve it, and release it with no human author. How much of this loop can safely run without a human? We propose two quantities a platform can measure. Verifier strength $\\sigma$ is how well the available verifier -- often an LLM judge -- agrees with a trusted reference verifier; it limits how much a step can improve the agent. Recoverability is how cheaply a bad change can be undone: whether undoing it needs a new verifier approval, and how much of the old behavior a rollback restores. A step should run unattended only if both are adequate. On our platform this rule allows prompt, tool and skill edits to run unattended and keeps weight updates under human sign-off.\n  Our main contribution is these two definitions and how to measure them. We built a platform that applies the rule and tested it in two ways: on the platform's own approval gate, and in controlled experiments on the $\\tau^2$-bench agent benchmark. On verifier strength, the platform's LLM judge sometimes gave the same score no matter what it was shown (9 of 37 settings) and reported no error, so a verifier can look healthy while approving anything; its strength must be measured, and two simple tests catch the failure. On recoverability, restoring a saved copy brought the agent back to its original behavior, but a regenerated undo did not, so only changes whose old version is kept can be reversed without a human; and while a change was deployed, it altered the outcome of 17 percent of tasks beyond normal run-to-run variation, which no undo repairs. Together these support the rule: a change can run unattended only when its old version is kept and its verifier's strength has been measured.", "url": "https://wpnews.pro/news/recoverability-and-verifier-strength-two-measured-preconditions-for-unattended", "canonical_source": "https://www.machinebrief.com/news/recoverability-and-verifier-strength-two-measured-preconditi-4zxe", "published_at": "2026-10-21 04:00:00+00:00", "updated_at": "2026-10-21 05:50:00.710442+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-research", "large-language-models"], "entities": ["arXiv", "τ²-bench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/recoverability-and-verifier-strength-two-measured-preconditions-for-unattended", "markdown": "https://wpnews.pro/news/recoverability-and-verifier-strength-two-measured-preconditions-for-unattended.md", "text": "https://wpnews.pro/news/recoverability-and-verifier-strength-two-measured-preconditions-for-unattended.txt", "jsonld": "https://wpnews.pro/news/recoverability-and-verifier-strength-two-measured-preconditions-for-unattended.jsonld"}}