# Hard Budget Caps Are a Runtime Safety Primitive for AI Agents > Source: > Published: 2026-10-04 05:31:35+00:00 An AI agent can fail without crashing, throwing an exception, or producing an obviously wrong answer. It can simply keep doing valid work for far too long. That is a different failure mode from the ones most software teams are used to. A retry loop can keep calling a paid API. A coding agent can keep launching test environments. A background worker can keep generating images, embeddings, logs, or storage objects. Every individual action may be permitted, yet the overall run becomes economically unsafe. This is why hard budget caps are starting to look less like billing features and more like runtime safety controls. Simon Willison made the case directly on October 3: usage-priced services need hard limits that stop work instead of merely sending an alert after a threshold is crossed. Google Cloud has moved in the same direction with Spend Caps, which can pause eligible services after a configured budget is exceeded. The important shift is architectural: cost is becoming something software can enforce, not merely something finance reviews later. For agent systems, that distinction matters even more because agents can create new work on their own. Traditional cloud budgets are usually observational. They answer questions such as: Those are useful questions, but none of them stop execution. An agent needs a control boundary. If a run is allowed to consume at most 500 units of a metered resource, then unit 501 should fail closed. The exact unit may be API credits, GPU minutes, database writes, browser actions, tokens, or a normalized internal cost score. The basic distinction is simple: ``` php soft budget: observe -> alert -> keep running hard budget: observe -> compare -> deny next action ``` The second model belongs on the execution path itself. Google Cloud's current Spend Caps preview makes this idea concrete. Its documentation says eligible service usage can be paused when the configured cap is enforced, while resources remain intact. Google also warns that enforcement is not instantaneous because estimated costs are involved, which is an important reminder: a provider-side cap is useful, but an application should still maintain its own tighter limits. A normal application usually has a fairly stable request shape. Users click buttons, jobs enter queues, and traffic rises or falls within recognizable patterns. Agents can create a feedback loop. A planner decides that a task failed. The retry policy creates another attempt. The new attempt spawns a browser session. The browser session calls an image service. The image service output fails validation. The planner decides to try again with a revised prompt. Nothing in that sequence is necessarily broken. The problem is multiplication. One bad decision can become twenty valid paid actions. A second recovery rule can turn those twenty into a hundred. If the only cost control is an email, the system can continue while nobody is watching. This is why an agent budget should be treated like a memory limit or a request deadline. It is not an accounting preference. It is a condition for continued execution. The safest place for a budget gate is immediately before a side-effecting or metered action. A simple implementation can keep the policy outside individual tools: ``` python from dataclasses import dataclass @dataclass class Budget: limit: int used: int = 0 def reserve(self, units: int) -> None: if units <= 0: raise ValueError("units must be positive") if self.used + units > self.limit: raise RuntimeError("hard budget exceeded") self.used += units budget = Budget(limit=500) def call_metered_tool(tool, estimated_units, **kwargs): budget.reserve(estimated_units) return tool(**kwargs) ``` This example is intentionally small. The important property is ordering: reserve first, execute second. If the budget check happens after the tool call, it is only telemetry. If a retry path can bypass the wrapper, it is not a hard cap. If a child agent receives a fresh counter instead of sharing the parent budget, spawning becomes a way to escape the limit. The budget object therefore belongs at the same architectural level as authentication, authorization, and cancellation. Money is not the only resource that needs a ceiling. A useful agent runtime can enforce several independent dimensions: ``` run_budget: max_tool_calls: 80 max_browser_writes: 12 max_external_api_units: 500 max_wall_clock_seconds: 1200 max_child_agents: 3 max_retries_per_step: 2 ``` These limits solve different problems. A wall-clock limit catches work that stalls without spending much. A tool-call limit catches rapid loops. A browser-write limit protects external accounts from repeated side effects. A retry limit prevents one failing step from consuming the entire run. A child-agent limit prevents recursive delegation from turning one task into an uncontrolled tree. The dimensions should be independent because a run can be safe in one dimension and dangerous in another. For example, a cheap API can still generate thousands of unwanted external objects. A browser automation task can cost almost nothing in infrastructure while repeatedly posting, liking, or modifying data. Financial cost alone would miss the real failure. Many metered operations do not reveal their final cost until they finish. That creates a race: several concurrent workers can all see remaining budget and begin expensive work at the same time. Reservations fix that. Before starting an operation, reserve the maximum or a conservative estimate. After it finishes, reconcile the reservation with actual usage. ``` python class Ledger: def __init__(self, limit): self.limit = limit self.committed = 0 def reserve(self, estimate): if self.committed + estimate > self.limit: return None self.committed += estimate return {"reserved": estimate} def settle(self, ticket, actual): reserved = ticket["reserved"] self.committed += actual - reserved ``` A production implementation needs atomic storage, but the model is what matters. Concurrency should not create free budget. This is similar to inventory systems. Two buyers should not both be able to purchase the last item because they read the same stock count before either write completes. Two agents should not both spend the same remaining allowance because they checked it at the same time. Provider-side limits are valuable because they are outside the application. If the application has a bug, the provider can still stop further billable usage. Application-side limits are valuable because they understand intent. A cloud provider sees requests. Your runtime sees tasks, users, sessions, tools, retries, and side effects. That suggests a layered design: Google Cloud's Spend Caps are an example of the first two layers for eligible services. The feature can pause usage after the configured threshold is reached, and Google documents alert points below the cap as well. But an agent runtime can act earlier because it knows whether a request is part of a retry storm or a legitimate new task. The two controls should reinforce each other rather than compete. Many agent systems treat every interrupted task as an error that deserves another attempt. That is dangerous when the interruption is the budget system doing its job. Budget exhaustion should be a first-class terminal state: ``` SUCCESS FAILURE BLOCKED CANCELLED BUDGET_EXHAUSTED ``` A scheduler receiving `BUDGET_EXHAUSTED` should not automatically retry the same plan with a fresh allowance. That would turn the budget into a delay instead of a boundary. The terminal record should include enough evidence to explain what happened: That makes recovery explicit. A human or higher-level policy can choose to grant more budget, reduce scope, or stop permanently. Retries are one of the easiest ways to accidentally defeat a cap. Suppose an image generation step fails quality validation. The system may reasonably try a second prompt. But if each retry receives the original full run allowance, the cost model is fictional. Retries should spend from the same parent budget and usually become stricter as attempts accumulate. One useful policy is: ``` retry_limits = { "network_timeout": 2, "rate_limit": 1, "validation_failure": 1, "authentication_failure": 0, "permission_denied": 0, } ``` The important part is not the exact numbers. It is that retry permission is based on failure class, and every retry still consumes the original run budget. This also improves behavior quality. An agent that knows it has one remaining attempt is more likely to gather evidence before acting than one that can keep trying indefinitely. Stopping spend should not mean destroying work. Google Cloud explicitly describes its Spend Caps as non-destructive: eligible usage can be paused while data and resources remain. Agent runtimes should follow the same principle. When a hard cap is reached: This produces a resumable system instead of a disposable one. It also separates two concerns that are often mixed together: stopping new risk and cleaning up old state. The first should happen immediately. The second should be deliberate and evidence-based. A prompt can tell an agent to be frugal. That is not enforcement. Prompts influence decisions. Runtime gates constrain actions. The safest architecture assumes that the model may misunderstand a limit, forget it after context compression, delegate to another process, or choose a plan whose cost estimate is wrong. A hard budget should still hold. That means the policy must live outside the model loop: ``` model proposes action | v policy checks permission | v budget reserves capacity | v tool executes | v ledger settles actual usage ``` The model can participate by estimating cost and selecting cheaper plans, but it should not be the final authority over whether the action is allowed. The broader lesson is that correctness for autonomous software has changed. A task is not correct merely because it eventually reaches the requested output. It also has to respect limits while getting there: time, external writes, retries, permissions, and spend. That is why the current push toward real spend caps matters. Simon Willison's argument for default hard limits is not only a billing recommendation. It points toward a runtime design principle for agentic systems: every autonomous loop needs a maximum amount of damage it is allowed to cause before the infrastructure stops it. Google Cloud's July 2026 Spend Caps preview shows that cloud platforms are beginning to expose this control at the billing layer. Agent frameworks should not wait for every provider to do the same. A reliable agent should know what it is allowed to spend, reserve that capacity before acting, share the same ledger across retries and child workers, and stop cleanly when the allowance is gone. That is not pessimism about autonomous systems. It is the mechanism that makes unattended execution practical. Sources: Originally published on [Dispatch](https://dispatch-blog.hashnode.dev/hard-budget-caps-are-a-runtime-safety-primitive-for-ai-agents).