Why Budget Alerts Never Stop Runaway LLM Spend

An engineer's blog post explains why budget alerts fail to prevent runaway LLM spend, citing a real incident where a $600 cap was exceeded by $1,300 due to a retry loop. The post argues that alerts are psychologically ineffective and proposes a reservation-based approach using database transactions to enforce budgets.

An "84% of monthly budget used" email landed on a Tuesday. I read it. I forwarded it to myself with a note that said watch this . Then I let the batch job run overnight anyway, and by Thursday morning the account had burned $1,900 against a $600 cap. Nothing malfunctioned. The alerting pipeline worked exactly as specified, delivered on time, to the right person, who understood it. That's the uncomfortable part, and it's not a story about discipline. Telling someone a resource is running low is one of the most-studied moves in behavioral psychology. The studied outcome is not restraint. It's an increase in perceived value and a pull toward consuming the remainder. Worchel, Lee, and Adewole ran the cleanest version of this in 1975, published in the Journal of Personality and Social Psychology , with 200 participants. Same cookie, same room, same everything, handed over from either a jar holding ten or a jar holding two. The two-cookie jar produced higher ratings on value and attractiveness. Then they added a disclosure condition: some participants were explicitly told the supply had dropped because other people wanted them. Being told the mechanism didn't cancel the effect. That group rated the cookie highest of all. Now read your dashboard banner again. "You have 16% of your budget remaining" is structurally identical to two cookies in a jar. It's a depletion cue with a full explanation attached, and the explanation is not a brake. You know the number is a threshold someone configured. You know the cap is arbitrary. Knowing changes what you can say about the banner. It does not change what you do next, which is usually to run the job. Here's the version almost everyone writes first, in some dialect: spent = usage store.month to date user id if spent BUDGET 0.9: log.warning "user %s at %.0f%% of budget", user id, spent / BUDGET 100 metrics.incr "budget.near limit" response = client.messages.create model="claude-sonnet-5", max tokens=4096, messages=messages, usage store.record user id, response.usage ledger updates after Every line of that is defensible in review. It still fails, in three specific ways. The ledger is always behind by exactly the calls that matter. record runs after the response returns. During a burst, the spend you most need to see is the spend that hasn't landed yet. Concurrency reads the same stale number. Twelve workers pull month to date inside the same second, all twelve see 92%, all twelve pass, all twelve fire. No single request overshot the cap. Collectively they cleared it by 4x. Agent loops move faster than any aggregator. A tool-calling agent that hits a malformed schema and retries will happily make forty calls in ninety seconds. If your usage rollup runs on a one-minute cron, the loop is finished before the graph even flinches. My $1,900 was mostly this: a retry loop with no ceiling, feeding a context window that grew on every pass. The common thread: that if statement doesn't gate anything. It observes and then gets out of the way. It's a very expensive log line. The fix is boring and it's the same one databases have used forever. Don't check the balance, hold against it. python class BudgetExceeded Exception : status code = 402 def reserve user id: str, estimated cost: float - Reservation: with db.transaction : row = db.query "SELECT spent, reserved, cap FROM budgets " "WHERE user id = %s FOR UPDATE", user id, .one if row.spent + row.reserved + estimated cost row.cap: raise BudgetExceeded f"{user id}: {row.spent + row.reserved:.2f} committed, " f"needs {estimated cost:.2f}, cap {row.cap:.2f}" db.execute "UPDATE budgets SET reserved = reserved + %s WHERE user id = %s", estimated cost, user id, return Reservation user id, estimated cost The call site becomes a hold, then a settle: estimate = price of model, count tokens messages , max tokens hold = reserve user id, estimate raises 402 before any network I/O try: response = client.messages.create ... hold.commit actual cost response.usage except Exception: hold.release raise Two details carry the weight. FOR UPDATE means those twelve concurrent workers serialize instead of all reading the same stale total, which kills the race that soft checks can't see. And the estimate is an upper bound computed from input tokens plus max tokens , so you reserve the worst case and refund the difference on commit. Over-reserving is a rounding error. Under-reserving is how you get a Thursday like mine. Most of the annoying parts here are already solved. baar-core https://github.com/orvi2014/Baar-Core is an open-source Python library that wraps this into a decorator around your LLM client: pip install baar-core , set a cap, and it raises a 402 before the provider is ever contacted rather than after the tokens are billed. Atomic reservation is the piece I'd have gotten wrong on my own, and it's the piece that stops concurrent calls from jointly clearing a limit that neither one individually exceeded. noburn.dev https://noburn.dev is what we built on top of it for teams: the same pre-flight enforcement blocking the API call before it fires when a user is over budget, plus per-user caps and the ledger view. Enforce at one boundary. A single wrapper around your client, not if statements scattered across seven call sites where the eighth one, added last month by someone in a hurry, has none. Reserve, don't observe. If your limit reads a number that a different process is responsible for updating, it isn't a limit. It's a lagging indicator wearing a limit's clothes. Fail typed and loud. Return a real 402 with a machine-readable reason so callers, retry middleware, and agent loops can all distinguish "out of budget" from "transient 500" and stop instead of backing off and trying again. A log line is advice. An exception is a wall. The reason this matters more than it should is the cookie jar. Alerts are built on the assumption that information changes behavior, and the research on depletion cues says information mostly changes your narration of the behavior. I didn't ignore my 84% email. I read it, understood it, and then did the thing anyway while explaining to myself why this run was different. Every person on your team will do the same, including the one who wrote the alert. So stop shipping warnings where you meant to ship a wall. What's the biggest gap you've measured between a spend alert firing and anything actually stopping? Originally published at https://robatdasorvi.com/stories/why-scarcity-works-even-when-people-know-it-is-fake