An agent's budget guard has one job: refuse the call that would take it past its ceiling. To do that it must hold the worst case before each call and settle the real cost after. The settling half is where guards quietly break, and they rarely throw an error when they do.
This week I found four of these in my own code. One was live, and undercharged. Two more would have undercharged, and one would have overcharged the user for something they had chosen not to buy.
When Claude streams, the first event, message_start, carries a usage block. Its output count is tiny: 1, 2 or 3 in Anthropic's own documentation examples. The real output count arrives later, in message_delta, and the docs are explicit: "The token counts shown in the usage field of the message_delta event are cumulative."
A tracker that folds usage as events arrive works perfectly, until the stream ends before message_delta. A proxy on a custom base_url that drops or reshapes it, a dropped connection, a regression after an upgrade. Then the call settles on the placeholder, and a 20,000-token reply is billed as one token. Nothing errors.
Fix: a stream is final only when a message_delta carrying an output count has arrived. Anything else is charged the worst case that was held for it.
record({}) settled a call at $0. An empty usage dict, a response with no usage, a key the parser skipped: all of them looked like a call that cost nothing.
Fix: usage without both an input and an output count is unreadable, and unreadable is charged the worst case, then raised. One check at the point where every provider's usage is settled, not a copy per provider, because the copies had already started to disagree.
The first draft gave the hold back on any 4xx: the API refused it, so surely nothing was billed. Then I went looking for the provider page that says so: the errors page, pricing, rate limits, streaming. None of them does. The only line I found was that a failed web search is not billed.
An assumption nobody can cite is not a ceiling. Fix: a call that raises is charged its worst case. If that ever overcharges someone, it is visible and recoverable. The other direction is how a limit gets breached.
To get final usage on a messages.stream() call, the obvious move is get_final_message(). In the Anthropic SDK that calls until_done(), which reads the rest of the stream. So a user who stopped a generation early, to save money, would be made to download and pay for the part they walked away from.
Fix: read usage from what has already arrived, never read further than the caller did, and charge the worst case if the final count never came.
A guard may be wrong, but only in the direction that protects the ceiling. Every one of these bugs was a guard being optimistic: trusting a placeholder, treating silence as zero, assuming an error was free. Optimism in a guard is a limit that stops holding without telling anyone.
I'm building Paveo, a checkpoint that runs inside your own process before every model and tool call. A budget is one rule it enforces; which models and tools an agent may use are others. Nothing leaves your machine. It works with Anthropic, OpenAI and Gemini today, and it's in a private alpha: if you run agents in Python and want to try to break it, say so in the comments.
A question for anyone running agents: besides cost, what would you want to block outright, before the call leaves?