cd /news/ai-safety/the-safety-layer-that-had-never-run · home topics ai-safety article
[ARTICLE · art-117171] src=dev.to ↗ pub= topic=ai-safety verified=true sentiment=· neutral

The safety layer that had never run

A developer building Pocket Change, a system that gives AI agents bounded spending authority, discovered that its most novel safety mechanism had never executed in real runs due to two stacked bugs. The developer fixed the issues by making sourcing data explicit and correcting the audit trail to distinguish between an absent monitor and a permissive one. The experience underscores the importance of instrumenting and verifying the mechanisms you are most confident in.

read4 min views1 publishedAug 31, 2026

I spent a week building a system whose entire argument is that an AI agent cannot

spend more than you authorised. Then, four days in, I found that the most novel

part of it had never executed. Not once.

Pocket Change gives AI agents bounded, auditable spending authority. You sign one

ceiling. A task decomposes into as many agents as it takes — 121 in one run — and

each one is handed a Biscuit capability token strictly narrower than its parent's.

Blocks are appended, never removed, so budget, tools and expiry all narrow together.

Raising any of them would mean forging a signature.

The idea I was proudest of is a separation. When a task says "find the best price",

something has to read pages written by sellers, which is where prompt injection

arrives. So the agent that reads the web holds search

and a budget of zero.

The agent that pays holds pay

and cannot search.

And crucially, the looker is not the payer's child. It's the payer's sibling.

That word carries the design. Attenuation is monotonic — a child holds a subset of

its parent — so hanging the searcher below the payer forces the payer to hold

search

for its own child to inherit it. That puts "reads hostile text" and "can

spend money" in the same token. I got this wrong once and shipped it, then fixed it

and wrote a test.

Days later I ran a real query, asked for the best price, and looked at what actually

happened. Every leaf had paid from the internasearched.

Two bugs, stacked, each individually invisible

First: sourcing reached the funnel only inside person's

answer was folded into prose — "find the best source; searching is allowed" — and

the root node was constructed at the default, ing model

read the sentence. The enforcement layer never saw it.

Second: the decomposer's output schema defaulted sourcing

to "catalogue"

. In the

funnel, inheritance was sub.sourcing or node.ue"

is

truthy. So every child declared a sourcing, and an explicit value beats

inheritance. A person's instruction reached thhing below it.

The mechanism I considered the intellectual cone I'd have

described first in any interview — had never run under the real decomposer. It only

ever appeared under a hardcoded fallback decomis configured.

Sourcing now travels as data, and follows the

EXPOSURE = {"catalogue": 0, "specific": 1, "best": 2}   # seller-written text

A sub-task inherits its parent's sourcing, may narrow it, and cannot widen
it. The untrusted decomposer can no longer walweb on its
own. Same monotonic principle, applied to the other axis of authority.

The second thing that wasn't true

While fixing that I checked the audit trail on a run with no API key configured:

"monitor": "allow",
"monitor_reason": "monitor not configured",
"monitor_ms": 0.094

With no key, the monitor falls back to a stand-in that allows everything. That is

the right failure — enforcement has already pamust not

block every payment. What was wrong was recording it as allow. A run with no

second layer read exactly like one that had pasole

checkbox saying the monitor was on.

An absent monitor is not a lenient monitor. It's an absent one. The audit now writes

monitor: unconfigured with monitor_ran: false,ws not

judged, and GET /status reports which layers are actually live.

What I'd tell anyone building agent infrastructure

Your tests can pass while the mechanism is dead. Mine did. 399 of them. Every

one exercised the funnel with a scripted decomin the seam

between the real decomposer's schema and the funnel's inheritance rule. Nothing on

either side of that seam was wrong on its own.

Instrument the thing you're proudest of. I fouhing

events on a live run and getting zero. Not by reading code — I'd read that code

many times.

A system that can't say what it did isn't trus

cryptography. The strongest thing in this project isn't the token chain. It's that

the audit trail will tell you a payment was ne

I created this post for the purposes of enteri Hackathon.

Pocket Change is open source: https://github.com/Somay-kousis/Pocket-Change

You sign one ceiling. The task splits into a tolding a

capability token cryptographically narrower than its parent's. No agent below can

widen what you signed — raising the cap would

The part I like most: the agent that reads seof ₹0 and

cannot pay. The agent that pays cannot search. They're siblings, not parent and

child — because a child inherits from its parxt" must

never share a token with "can spend money".

Then a critic reads the plan before any authority is minted. On one run it refused

a branch that every arithmetic bound had passhorized

purpose of furnishing the new engineering office."

The money was correct. The plan had drifted.

399 tests, all offline. Gemini 3.5 Flash on Vertex AI, Cloud Run, Firestore.

── more in #ai-safety 4 stories · sorted by recency
── more on @pocket change 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/the-safety-layer-tha…] indexed:0 read:4min 2026-08-31 ·