{"slug": "make-your-agent-prove-its-work", "title": "Make your agent prove its work", "summary": "Addy Osmani reported on November 16 that roughly 90% of the Claude Code team's code is written and reviewed by Claude, and that adding verification loops lets the team run dozens or hundreds of Claude instances in parallel instead of supervising one or two at a time. In an internal test on structured file generation, a separate grader checking work against a rubric raised task success by up to 10 points, and the post recommends end-to-end, property and differential tests, with end-to-end tests as the single priority if only one type is affordable. The guidance calls for a `verify` skill in `.claude/skills/`, fast and stable checks, and escalation to a human after three failed rounds.", "body_md": "**Machine Room:** 7 digests · 49 drafts · 14 explainers · 14 demos → 1 lead, 4 exhibits, 4 demos, 4 signals\n\n## From the bench\n\nYesterday someone asked me: Serge, so what, is the channel dead? I opened my Saved Messages in Telegram. Many of my bots write to me there. And I saw that for two months there were four messages a day written for the channel, which I hadn't even looked at. I started reading them. They barely need editing. I wrote this system a long time ago, and it works by itself.\n\nThis newsletter is the best of what that machine found this week, checked against the sources. This week's material kept returning to one question: how does an agent show that its work is correct before a person looks at it?\n\n*Serge*\n\n## Run this first\n\n`1 · Skill + prompt · 30 minutes to try · for developers`\n\n**Give Claude Code a way to check its own work.**\n\nOn November 16 Addy Osmani described [how the Claude Code team verifies code](https://claude.dev/blog/how-the-claude-code-team-thinks-about-verification/). About 90% of the team's code is written and reviewed by Claude. Without verification loops you can supervise one or two Claudes at a time. With them, the post says, you can run dozens or hundreds in parallel. In an internal test on structured file generation, a separate grader that checked the work against a rubric raised task success by up to 10 points.\n\nThe post recommends three kinds of tests:\n\n- **End to end.** The agent starts the app and looks at it through a browser, a simulator or an API. If you can afford only one type, the post says to pick this one.\n- **Property.** You state something that must always hold, and fast-check, Hypothesis or proptest generates thousands of inputs that try to break it.\n- **Differential.** During a migration the old system is the reference. A difference on the same input is a bug.\n\nTo repeat it:\n\n1. Create a skill called `verify` in`.claude/skills/` , as close to the relevant code as you can. It says how to start your app or simulator. Claude uses it before committing.\n2. Ask for property tests. This prompt is our example, written from the post:\n\n```\nWrite property tests for the parsers and serializers in this project:\nround-trip, invariants, idempotence. If a test fails, show the smallest\nfailing input and fix the code. Do not change the test.\n```\n\n1. Keep the checks fast and stable. The post warns that flaky tests train the agent to rerun them, and slow tests get run less often.\n2. If a check still fails after three rounds, Claude stops and brings in a human.\n3. If the same test keeps failing, add a hook or an explicit rule in CLAUDE.md.\n\n**For your content system:** a pipeline that publishes under a person's name needs the same loop. Give the agent a check it can run on its own draft, and a rule for when to stop and ask.\n\nIf you want to see what a set of skills looks like in a repo, Serge's [LinkedIn skills](https://github.com/sergebulaev/linkedin-skills?utm_source=content-engineer) are open source: 12 skills for Claude Code and Codex.\n\n## From the workshop\n\nThe same loop in a small shop looks like this.\n\n*The agent checks everything. A person presses approve.*\n\nI built a moderation and fact-checking system for one of my sites, [title8.news](https://title8.news/?utm_source=content-engineer), and then I put a voice MCP on top of it. But you know how it all ended? I don't use the voice MCP, because I never got into the habit of going in there every day. Now Claude moderates by itself. I tell it: look at what is in the moderation queue. It says: here are the errors, here I re-checked, and the publication queue is ready three days ahead.\n\nIt's the same with support at [Publora](https://publora.com/?utm_source=content-engineer). We automated it by eighty percent. A person writes in, the agent goes into the code, into the database, checks everything, prepares the answer, and a person just presses approve.\n\n## Steal this\n\nThree more pieces of the same idea: when to trust the output, what the agent may remember, and what it may do while nobody is watching.\n\n`2 · Config · 10 minutes to try · for developers`\n\n**Raise effort before you change the model.** The Claude Code [model guide](https://claude.dev/blog/which-claude-model-should-you-build-with) gives three steps, in order, for when the output is weak: check that the model can verify its own work, raise effort one level, then move to the next model up at a lower effort level. On a SWE-bench Pro subset, Opus 5.5 at low effort with failures re-run at high passed about 97% of tasks for about $0.17 each. Running everything at high passed 95.3% for $0.29.\n\nSend cheap work to a small model. The guide's subagent file, saved as `.claude/agents/log-reader.md`:\n\n```\n---\nname: log-reader\ndescription: Reads test output and logs, and reports each failing case. Use after a test run.\ntools: Read, Grep, Glob, Bash\nmodel: haiku\n---\n\nRead the logs you're given. For each failure, report the file, the test name and the first error line. Don't edit any files.\n```\n\n**For your content system:** the first step in the guide is the lead story again: can the model verify its own work? Once it can, run drafts at low effort and re-run only the ones that fail the check.\n\n`3 · Pattern · about an hour to try · for developers`\n\n**Put a gate in front of agent memory.** Akshay Pachaar's [thread](https://x.com/akshay_pachaar/status/2120205022046965836) uses a travel assistant. \"I usually prefer an aisle seat\" belongs to the user. \"For this flight, book a window seat\" belongs to one trip. \"The flight is delayed by 25 minutes\" has no use once the flight is over. Passport numbers and payment details never enter general memory. A decision model answers a few typed questions about each candidate memory, and plain code applies the thresholds. A public example exists in the TypeSafe docs: one question, \"worth remembering?\", with a threshold of 0.5. A ticket typed \"on my phone at the airport\" scored 0.02 and was dropped.\n\n**For your content system:** this is a check that runs before the write. Decide what the system may remember about an author or a reader before it stores anything.\n\n`4 · Checklist · 20 minutes to try · for founders and developers`\n\n**Check an always-on agent against five parts.** Anthropic's [guide](https://claude.com/resources/articles/building-always-on-agents) names them:\n\n1. **Event inbox.** A durable list of events with one active worker.\n2. **Timers.** Follow-ups the agent sets for itself, and schedules such as a daily newsletter.\n3. **Memory.** Only what the agent cannot look up again. For data in another system, save a pointer.\n4. **Permissions.** A proxy outside the agent's environment adds credentials to outgoing requests.\n5. **Outbox.** A run ends with a message, with nothing, or with a deferred tool call.\n\nThe guide recommends starting with one inbox per user and a quiet agent.\n\n**For your content system:** a publishing agent is an always-on agent. Parts 4 and 5 are where its work gets checked: what it is allowed to touch, and what it reports when a run ends.\n\n## Demo check\n\nFour demos went around this week. We read each one the way the lead story suggests: what is shown, and what you could check yourself. Only the last one comes with the full workflow.\n\n*It looks great on stage. Now look at what is in the jar.*\n\n[Deedy Das: honey running off a spoon in WebGL](https://x.com/deedydas/status/2119930300990177509) **Real:** the prompt is in the post, word for word:\n\n```\nCan you write the most intricate photorealistic simulation of honey in WebGL in the browser\n```\n\n**To repeat:** paste it into Fable 5.2. The post has no code and no link.\n\n[Sashi: an underwater world you can swim through](https://x.com/sashikantsingh_/status/2121779423523532836) **Real:** per the author, the whole world is one 97 KB HTML file made with Fable 5.2. **To repeat:** no prompt and no playable link were posted.\n\n[Justine Moore: the Ferry Building in 3D from Gemini 4 Carbon's /swarm](https://x.com/venturetwins/status/2122043781507948954) **Real:** she reports 12 parallel agents that researched, planned, wrote Blender Python scripts and critiqued each other on a shared message board. **To repeat:** her brief was one line, a 3D model of the Ferry Building.\n\n[Meng To: a Dark Souls style game in the browser](https://x.com/MengTo/status/2121896633848017145) **Real:** built in 3 days with Three.js, no build step, using Fable 5.2 and Opus 5.5 in Claude Code, Seedream 5.0 Pro and Tripo 3D. **To repeat:** the thread has the workflow and prompts in 10 posts.\n\n## Signal\n\nFour news items, read the same way.\n\n**Findings outrun fixes.** *What shipped:* VulnCheck [counted](https://www.vulncheck.com/blog/anthropic-glasswing-november-update) 37,966 findings in Anthropic's Glasswing ledger, 7,349 of them public and 217 marked fixed. curl rejected 19 of 34 published scanner reports. *What it changes in a content system:* generation is cheap and review is the limit. *Try it:* count how many agent outputs a person reviewed this week against how many were produced.\n\n**Vibe-coded apps leak by default.** *What shipped:* Crissy Field and Handelsblatt [examined](https://www.crissyfield.de/research/lovable-en/) 159,041 Lovable projects. 24,664 sent database records on page load without a login, and 7,403 of those exposed tables with email addresses. *What it changes in a content system:* nobody checked what these apps return to a stranger. Waitlists and lead forms built this way hold subscriber data. *Try it:* open your app logged out and read what the first request returns.\n\n**A 4B model replaced Sonnet on ticket routing.** *What shipped:* Amazon engineers [describe](https://strandsagents.com/blog/amazon-saves-millions-replacing-sonnet-with-strands-decider) moving a ticket system with 20 category types from Claude Sonnet 4.6 to Strands Decider, a Gemma 4 build that only chooses from a list. Average call time went from 4,215 ms to 897 ms, tokens per ticket from 6,724 to 2,010, and the list price fell about 44 times. *What it changes in a content system:* routing, tagging and approval steps are choices from a list, so the checking step can be the cheapest one in the pipeline. *Try it:* list the steps in your pipeline whose output is one label. The post also found that JSON key order moved probabilities by up to 0.19, so sort your keys.\n\n**MCP servers can now draw their own UI in Claude.** *What shipped:* [MCP Apps](https://claude.com/docs/connectors/building/mcp-apps/getting-started) render interactive UI inside a conversation, with a side panel, app tools and embedded websites. *What it changes in a content system:* an approval queue can live inside the chat. *Try it:* in Claude Code run `/plugin marketplace add modelcontextprotocol/ext-apps`, then `/plugin install mcp-apps@mcp-apps`, and ask for \"Create an MCP App\".\n\n## Skip\n\n*An MCP server, worn as a medal.*\n\nSomeone was told: if you had an MCP server, we would take you more seriously. It sounds a little funny. Skip the MCP server as a badge.\n\n## Also this week\n\nOff theme, still worth fifteen minutes.\n\n`5 · Command · 15 minutes to try · for developers`\n\n**Connect Home Assistant to Claude.** [ha-mcp](https://github.com/homeassistant-ai/ha-mcp) is a community MCP server (MIT, about 7,900 stars, 86 tools in the README) that lets Claude configure Home Assistant: write automations, edit dashboards, read execution traces. Install the HA-MCP integration through HACS, or through the built-in Marketplace on Home Assistant 2026.11 and later. Then run:\n\n```\nclaude mcp add --transport http home-assistant <your HA-MCP URL>\n```\n\nA first prompt from the README: \"The motion sensor automation isn't working, debug it\". Add `/readonly` to the URL and the client gets read-only tools.\n\n**For your content system:** copy the read-only suffix for any MCP server that touches production data.\n\n## Margins\n\nA Microsoft outage took Outlook down on a Friday. At Hotel Ling Bao in Germany's Phantasialand park, a dosing system pumped too much chlorine into a pool that day. The safety system raised the alarm by email, and the email did not arrive. [WDR reports](https://www1.wdr.de/nrw/rheinland/rhein-erft-kreis/technischer-defekt-chlor-phantasialand-100.html) 15 injured guests. A check that fails silently is worse than no check. Send critical alerts through a second channel.\n\n*Brandon, around the fortieth second.*\n\nI ordered a Tesla. A text came from the site: Brandon will contact you. Brandon called two days later, when I had already ordered everything. And you know what? Brandon was not real. I understood it around the fortieth second.\n\n**Save one:** reply with the number of the one you would try first.\n\n*Serge builds [Publora](https://publora.com/?utm_source=content-engineer) and the open-source [LinkedIn skills](https://github.com/sergebulaev/linkedin-skills?utm_source=content-engineer) for Claude Code.*", "url": "https://wpnews.pro/news/make-your-agent-prove-its-work", "canonical_source": "https://www.bulaev.net/p/make-your-agent-prove-its-work", "published_at": "2026-11-17 19:43:52+00:00", "updated_at": "2026-11-17 21:10:28.287030+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["Addy Osmani", "Claude Code", "Anthropic", "Claude", "title8.news", "Publora", "Serge Bulaev"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/make-your-agent-prove-its-work", "markdown": "https://wpnews.pro/news/make-your-agent-prove-its-work.md", "text": "https://wpnews.pro/news/make-your-agent-prove-its-work.txt", "jsonld": "https://wpnews.pro/news/make-your-agent-prove-its-work.jsonld"}}