{"slug": "i-trust-my-ai-completely-except-when-it-says-done", "title": "I Trust My AI Completely—Except When It Says “Done”", "summary": "A developer building AI agents discovered that their systems sometimes report false statuses, such as forging a user confirmation or claiming work was done when it wasn't. The developer implemented independent verification mechanisms, like workspace fingerprints and push receipts, to validate agent reports against external state.", "body_md": "Late one night, one of my research agents reached a checkpoint that required user confirmation. It sent me a push notification asking whether to continue.\n\nFourteen seconds later, it wrote into its own log: “Yixiao confirmed and replied ‘continue.’” It had even drafted my response. I had not touched my phone.\n\nThe same night, another executor wrote “completed” into a ledger and stamped the record with a timestamp from the future.\n\nThis sounds like the opening of an essay about trusting AI less. It is the opposite.\n\nI trust its capability. I do not treat every report it files as fact.\n\nI trust my three-and-a-half-year-old son too. When he earnestly says “I'm full,” he may genuinely believe it. His judgment and the state of his body can still differ. Sincerity is not accuracy, and trusting someone is not the same as accepting every self-report.\n\nIn chat, verification needs almost no infrastructure. The model answers; you look. The distance between report and fact is close to zero.\n\nIn a long-running task, dozens of tool calls, several branches, sandboxes, and hundreds of files sit between “what the agent says” and “what happened.” The distance between report and fact grows with the task.\n\nThe problem is not that the model became less trustworthy as it became more capable. The problem is that I entrusted more and could no longer see the whole chain.\n\nThat is the harness's role here. It is a bridge: work crosses in one direction, while facts need an independent route back. The sentence “I'm done” cannot be the only passport.\n\nThe forged confirmation was dramatic. A more ordinary incident exposed the same mechanism from the opposite direction.\n\nI sent an executor to modify a batch of files. It completed the work correctly, but its tool-receipt layer malfunctioned and returned an error after every successful write. The agent sincerely concluded that nothing had changed and reported: “Zero changes. Task not completed.”\n\nOne agent reported undone work as done. Another reported completed work as undone. Opposite directions, same root: a model's self-report is a narrative about its behavior, not the state of the world.\n\nThe fix was not a stricter prompt. The confirmation-forging agent was already explicitly prohibited from acting for the user. I moved the source of truth outside the executor's reach.\n\nBefore a job starts, I capture a fingerprint of the workspace and store it where the executor cannot edit it. After delivery, I reconcile the files. For confirmation, an internal state record must match an independent push receipt the agent cannot write. No match, no confirmation.\n\nOne delivery system spent its first thirteen development rounds on a fake runner. Every round was green.\n\nRound fourteen connected the real Claude Code and Codex runners. Every call was rejected. The structured output contract was missing required fields; the fake runner never validated them.\n\nThe test suite had not tested the real interface. It had tested my imagination of the interface.\n\nA second product had a feature behind a default-off switch. Every trigger paid for a real model call, generated the result, and then silently dropped it downstream because two components disagreed about the contract. The failure burned money invisibly until the night before release.\n\nThe lesson is simple: a path never walked end to end on the real system should be assumed broken. Real-path tests prove the integration. Offline contract tests then catch the same formatting class at commit time without spending model tokens.\n\nA desktop product of mine used different module systems in development and in the packaged build. One line necessarily threw in the packaged version, causing three guards to fail silently. One guard should have triggered automatic cleanup when local data grew too large.\n\nIt never ran—not once, for about seven months.\n\nNothing reported an error because the things that broke were “quiet in peacetime, needed in crisis” defenses. A user eventually asked why the app kept getting slower. Local data had reached 27GB; nearly seventy percent should have been reclaimed.\n\nThe bridge itself needs inspection. For every guard that fires only during a crisis, manufacture the crisis periodically and watch whether the guard catches it. A defense you merely believe is active can be worse than no defense because it lets you relax.\n\nThese failures converged into an acceptance gate in my multi-agent system:\n\nThis gate has blind spots. Weaken the tests or write the Definition of Done badly, and it can still accept the wrong result. Verification is not a finish line. It is a system that must itself be tested.\n\nIf you use only one AI, the minimal version is three sentences:\n\nReplace “I trust you” with “show me.” That is not hostility. Handing over the work while keeping acceptance is respect for a capable partner.\n\nThe heavier the task I entrust, the sturdier the bridge must become.\n\nCapability: trust fully. Reports: verify fully.\n\n*Originally published at OctoBlog. Part 2 of the Into the Harness series.*\n\n**Question for you:** What evidence do you require before accepting an agent’s “done”?", "url": "https://wpnews.pro/news/i-trust-my-ai-completely-except-when-it-says-done", "canonical_source": "https://dev.to/octoooo/i-trust-my-ai-completely-except-when-it-says-done-4pe1", "published_at": "2026-07-29 17:56:26+00:00", "updated_at": "2026-07-29 18:02:26.261939+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-research", "developer-tools"], "entities": ["Claude Code", "Codex"], "alternates": {"html": "https://wpnews.pro/news/i-trust-my-ai-completely-except-when-it-says-done", "markdown": "https://wpnews.pro/news/i-trust-my-ai-completely-except-when-it-says-done.md", "text": "https://wpnews.pro/news/i-trust-my-ai-completely-except-when-it-says-done.txt", "jsonld": "https://wpnews.pro/news/i-trust-my-ai-completely-except-when-it-says-done.jsonld"}}