{"slug": "the-safety-layer-that-had-never-run", "title": "The safety layer that had never run", "summary": "A developer building Pocket Change, a system that gives AI agents bounded spending authority, discovered that its most novel safety mechanism had never executed in real runs due to two stacked bugs. The developer fixed the issues by making sourcing data explicit and correcting the audit trail to distinguish between an absent monitor and a permissive one. The experience underscores the importance of instrumenting and verifying the mechanisms you are most confident in.", "body_md": "I spent a week building a system whose entire argument is that an AI agent cannot\n\nspend more than you authorised. Then, four days in, I found that the most novel\n\npart of it had never executed. Not once.\n\nPocket Change gives AI agents bounded, auditable spending authority. You sign one\n\nceiling. A task decomposes into as many agents as it takes — 121 in one run — and\n\neach one is handed a Biscuit capability token strictly narrower than its parent's.\n\nBlocks are appended, never removed, so budget, tools and expiry all narrow together.\n\nRaising any of them would mean forging a signature.\n\nThe idea I was proudest of is a separation. When a task says \"find the best price\",\n\nsomething has to read pages written by sellers, which is where prompt injection\n\narrives. So the agent that reads the web holds `search`\n\nand a budget of **zero**.\n\nThe agent that pays holds `pay`\n\nand cannot search.\n\nAnd crucially, the looker is not the payer's *child*. It's the payer's *sibling*.\n\nThat word carries the design. Attenuation is monotonic — a child holds a subset of\n\nits parent — so hanging the searcher below the payer forces the payer to hold\n\n`search`\n\nfor its own child to inherit it. That puts \"reads hostile text\" and \"can\n\nspend money\" in the same token. I got this wrong once and shipped it, then fixed it\n\nand wrote a test.\n\nDays later I ran a real query, asked for the best price, and looked at what actually\n\nhappened. Every leaf had paid from the internasearched.\n\nTwo bugs, stacked, each individually invisible\n\nFirst: sourcing reached the funnel only inside person's\n\nanswer was folded into prose — \"find the best source; searching is allowed\" — and\n\nthe root node was constructed at the default, ing model\n\nread the sentence. The enforcement layer never saw it.\n\nSecond: the decomposer's output schema defaulted `sourcing`\n\nto `\"catalogue\"`\n\n. In the\n\nfunnel, inheritance was `sub.sourcing or node.ue\"`\n\nis\n\ntruthy. So every child *declared* a sourcing, and an explicit value beats\n\ninheritance. A person's instruction reached thhing below it.\n\nThe mechanism I considered the intellectual cone I'd have\n\ndescribed first in any interview — had never run under the real decomposer. It only\n\never appeared under a hardcoded fallback decomis configured.\n\nSourcing now travels as data, and follows the\n\n```\nEXPOSURE = {\"catalogue\": 0, \"specific\": 1, \"best\": 2}   # seller-written text\n\nA sub-task inherits its parent's sourcing, may narrow it, and cannot widen\nit. The untrusted decomposer can no longer walweb on its\nown. Same monotonic principle, applied to the other axis of authority.\n\nThe second thing that wasn't true\n\nWhile fixing that I checked the audit trail on a run with no API key configured:\n\n\"monitor\": \"allow\",\n\"monitor_reason\": \"monitor not configured\",\n\"monitor_ms\": 0.094\n```\n\nWith no key, the monitor falls back to a stand-in that allows everything. That is\n\nthe right failure — enforcement has already pamust not\n\nblock every payment. What was wrong was recording it as allow. A run with no\n\nsecond layer read exactly like one that had pasole\n\ncheckbox saying the monitor was on.\n\nAn absent monitor is not a lenient monitor. It's an absent one. The audit now writes\n\nmonitor: unconfigured with monitor_ran: false,ws not\n\njudged, and GET /status reports which layers are actually live.\n\nWhat I'd tell anyone building agent infrastructure\n\nYour tests can pass while the mechanism is dead. Mine did. 399 of them. Every\n\none exercised the funnel with a scripted decomin the seam\n\nbetween the real decomposer's schema and the funnel's inheritance rule. Nothing on\n\neither side of that seam was wrong on its own.\n\nInstrument the thing you're proudest of. I fouhing\n\nevents on a live run and getting zero. Not by reading code — I'd read that code\n\nmany times.\n\nA system that can't say what it did isn't trus\n\ncryptography. The strongest thing in this project isn't the token chain. It's that\n\nthe audit trail will tell you a payment was ne\n\nI created this post for the purposes of enteri Hackathon.\n\nPocket Change is open source: [https://github.com/Somay-kousis/Pocket-Change](https://github.com/Somay-kousis/Pocket-Change)\n\nYou sign one ceiling. The task splits into a tolding a\n\ncapability token cryptographically narrower than its parent's. No agent below can\n\nwiden what you signed — raising the cap would\n\nThe part I like most: the agent that reads seof ₹0 and\n\ncannot pay. The agent that pays cannot search. They're siblings, not parent and\n\nchild — because a child inherits from its parxt\" must\n\nnever share a token with \"can spend money\".\n\nThen a critic reads the plan before any authority is minted. On one run it refused\n\na branch that every arithmetic bound had passhorized\n\npurpose of furnishing the new engineering office.\"\n\nThe money was correct. The plan had drifted.\n\n399 tests, all offline. Gemini 3.5 Flash on Vertex AI, Cloud Run, Firestore.", "url": "https://wpnews.pro/news/the-safety-layer-that-had-never-run", "canonical_source": "https://dev.to/casperday11/the-safety-layer-that-had-never-run-27n1", "published_at": "2026-08-31 23:42:31+00:00", "updated_at": "2026-08-31 23:52:51.002719+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "developer-tools"], "entities": ["Pocket Change"], "alternates": {"html": "https://wpnews.pro/news/the-safety-layer-that-had-never-run", "markdown": "https://wpnews.pro/news/the-safety-layer-that-had-never-run.md", "text": "https://wpnews.pro/news/the-safety-layer-that-had-never-run.txt", "jsonld": "https://wpnews.pro/news/the-safety-layer-that-had-never-run.jsonld"}}