# Billing an AI Agent Without Breaking Its Tool Loop > Source: > Published: 2026-09-24 02:20:49+00:00 A user asks a desktop agent to prepare a promotion: inspect a few products, check inventory, draft the copy, and generate a candidate image. Publishing still requires the usual business approval. This is one user request. It may involve several model requests, local tool calls, and a separate image-generation request. Now put a subscription gateway between the agent and its model provider. The gateway might count every model request as a new conversational “turn.” Or it might withhold a finished result until its billing record settles. The agent stops halfway through an otherwise valid workflow, even though the account still has allowance. The failure looks like an agent problem. The underlying mistake is that adding metering has changed who controls the work. This article examines that boundary: how to add account-level model billing while leaving the agent's context, tool loop, and execution decisions in its host. The examples are illustrative; the implementation discussed at the end is publicly inspectable. For a locally orchestrated agent, the host owns the conversation and the loop around model calls. It assembles context, supplies tool schemas, interprets responses, executes authorized tools, and decides whether another model request is needed. A billing gateway has a narrower responsibility: authenticate the caller, check access to the requested model service and its allowance, forward the request, preserve the result, and account for usage. ``` php Local agent host | | model request v Model gateway --> Model provider | | streamed response / stored result v Local agent host | | validated tool call, under the original authorization v Business execution path --> Business system | | business result v Local agent host --> next model request, if needed ``` The same platform may provide both the model gateway and the governed business execution path. They still make different decisions. Paying for model access does not grant permission to change a product's price. A hosted workflow engine can be a valid architecture too. The important question is whether orchestration was deliberately assigned to it. Introducing a billing service should not silently create a second planner, a second conversation, or a second tool loop. This also does not promise that a proxied connection has identical latency or supports every provider feature. It adds a network hop, authentication, and persistence. Those costs deserve measurement. They do not require taking ownership of the user's task. “Request” is an overloaded word in an agent system. | Lifetime | Example | What its identifier is for | |---|---|---| | User turn | “Prepare this promotion” | Conversation and UI correlation | | Model operation | One inference request or one image-generation request | Model result recovery and metering | | Business invocation | Change one approved product description | Authorization, execution evidence, and business recovery | A single turn can contain many model operations. A model response can propose several tool calls. Some local tools may consume no provider usage at all. The host therefore creates a new model-operation ID for a genuinely new model request, and persists it before sending. If that request loses its response, recovery uses the original ID. Another user message is not a reason to replace an unresolved operation's identity. Idempotency also needs a scope. In the implementation described here, model-operation identity is scoped to the authenticated user and usage account. Optional conversation and turn coordinates provide correlation; they are not the billing key. Reusing the same scoped ID with different request content is a conflict. Business writes keep their own invocation identities. A model-operation receipt cannot prove that the resulting inventory update happened, and a business invocation ID cannot reconstruct a missing model response. For example: model operation A proposes an edit, the host submits business invocation X under the existing approval rules, and model operation B interprets X's result. These three records describe related work, but none can stand in for the others. Two questions need separate answers: A useful image can arrive without enough metering information to settle its charge. The image is available; accounting is unresolved. Collapsing both facts into a single `pending` flag makes the application behave as though generation never finished. Conversely, accepting a billable request does not mean a result exists yet. Consider this illustrative image-operation timeline: ``` t0 Persist operation and pricing snapshot; dispatch once t1 Return accepted/pending with the original operation ID t2 Receive and durably save the image result t3 Host retrieves the image; it may continue its local workflow t4 Settle the charge when saved metering is sufficient If metering is insufficient at t4: keep billing pending; do not hide the image or generate it again ``` Durable persistence still matters. Moving settlement off the response path does not make the gateway independent of its database. The implementation must preserve enough result and metering evidence to recover after a crash. A storage failure is a real failure, not something to conceal behind an optimistic success message. For chat streaming, deliver provider events as they arrive instead of waiting for a final ledger calculation. The host's provider adapter still decides whether the response is usable: complete tool arguments, finish conditions, provider errors, and truncated output remain significant. A text preview can appear before the operation's final receipt; executing a tool requires the completed response and the host's execution checks, not a fragment of streamed arguments. In particular, a gateway can finish receiving an HTTP error response. Its result envelope is available, but transport completion establishes neither inference success nor proof that no execution occurred. A transport-level `complete` state must never be promoted directly to “the task succeeded.” Image adapters and raw chat forwarding can have different completion semantics; document both. A pending bill for a completed operation should not automatically block every unrelated operation in the account. That choice improves continuity, but it also creates a financial exposure that needs an explicit policy. One simple policy admits requests while the account's settled balance is positive, then deducts actual charges asynchronously. Once settled charges exhaust the allowance, new requests are denied. Already-admitted output is not cut off merely to enforce settlement in real time. This is a soft admission boundary. Suppose an account has $0.30 remaining. Two concurrent requests both pass admission, and each eventually costs $0.25. They consume $0.50, leaving $0.20 of overage. A correct ledger records that overage rather than dropping part of the charge. More importantly, asynchronous accounting does **not** guarantee that overage stays small. High concurrency, expensive requests, or persistently missing usage can increase exposure. If pending operations remain unpriced, a positive settled balance can continue admitting work. An operator who needs a hard spending ceiling needs a different design: trustworthy cost bounds, reservations, limits on outstanding liability, or some combination. That can reduce concurrency or reject a request before its final cost is known. It is a product and risk decision, not a rounding detail. Whichever policy you choose, explain the constraint in its actual unit. A maximum number of model calls is not interchangeable with a dollar allowance. A task-level write budget is another independent control: it limits business activity, not model spending. Resource protection still applies. Timeouts, bounded response storage, and provider limits can interrupt a request. “Billing does not interrupt streaming” is a much narrower claim than “streaming cannot fail.” If the connection drops after dispatch, the provider may already have generated an image or consumed inference resources. The first recovery action should be a read of the original operation record. It should not submit another generation request under a new ID, switch providers, or change the model and hope for the same answer. There are several distinct situations: | What is known | Appropriate next step | |---|---| | Request was rejected before provider dispatch | Fix the stated admission or configuration problem | | Original operation is still pending | Continue inspecting that operation | | Stored result is available | Let the appropriate host adapter handle it, independently of settlement | | Execution outcome is unknown | Preserve its identity and uncertainty; do not automatically redispatch | | Stored response has expired | Report the retention limit; expiration does not prove non-execution | This mechanism prevents duplicate dispatch attempts within the gateway's idempotency boundary. It is not an end-to-end exactly-once guarantee. A crash after the provider accepts work but before the gateway saves its result can still leave an uncertain outcome. Reading a durable gateway record is also different from querying a provider's asynchronous job API. Without a provider job ID and a supported query mechanism, the gateway cannot promise to recover a result it never received. Likewise, a settlement worker cannot manufacture missing usage. Recomputing a bill from saved metering only helps when that metering is sufficient. Cancellation has the same boundary. The host can stop using an operation's result and end the local turn. That does not prove the provider stopped processing or incurred no cost. Available late usage must still be recorded, while a late result must not reopen the cancelled turn or trigger its tools. Text and image services do not necessarily report usage in the same unit. Preserve the provider's actual measurements. An image adapter that receives image counts should not invent token counts to make a dashboard look uniform. A shared allowance can instead use supported usage dimensions and their configured unit prices: ``` reference charge = sum(actual metered quantity × reference unit price) account debit = reference charge × plan multiplier ``` Those quantities must match the pricing rules. Missing or unsupported measurements need an explicit pending or unavailable state, not an assumed zero. Public price catalogs can help maintain reference rates. They do not establish what another provider charged you, and matching similar model names is not enough to prove equivalence. Keep the execution model mapping explicit, record the chosen reference model and endpoint, and retain the price and multiplier snapshot for each operation. Historical charges should remain explainable after prices change. Likewise, a late completion should settle against the allowance pool that admitted it, not quietly consume a newly reset pool. The customer may see Credits or a remaining percentage. That presentation does not change the underlying measurements. For periodic plans, the percentage denominator is the period's allowance, which may differ from the subscription sale price. A stub provider makes these boundaries observable. Count its invocations, control when it emits bytes, and inspect the persisted request and ledger records. These are useful acceptance checks; they are not a claimed turnkey test harness: | Injected condition | Check | |---|---| | One user message requires several model requests | The host keeps control of the loop; no arbitrary turn counter replaces the allowance policy | | Slow first chunk or slow final usage | Streaming and result availability do not wait for financial aggregation | | Image result arrives without usable metering | Image remains available; billing stays explicitly pending | | Same operation ID is submitted concurrently | One dispatch attempt; conflicting content under that ID is rejected | | Provider accepts work, but the acknowledgement is lost | Recovery reads the original operation; provider invocation count does not increase | | User cancels before a late response | Usage can still be recorded; the finished local turn stays closed | | Two requests spend the last positive balance | Actual overage is retained; further admission follows the documented policy | | Allowance resets before an old request settles | The old charge stays attached to its original pool | Measure latency separately from accounting correctness. Time to first byte, time to a host-usable result, and time to settlement describe different parts of the experience. A single “request duration” metric can hide the very coupling you are trying to remove. I maintain BailingHub, an open-source project for governed access to business capabilities. Core **0.9.0**, Agent Client SDK **0.7.0**, and the paired DSH plugin **0.7.0** provide an optional model billing gateway along these lines. The published contract declares `orchestration=host` and `turn_required=false`. It separates chat-model and model-tool catalogs, model-operation results and settlement, and raw usage and customer-facing allowance. Its admission policy uses the positive settled balance described above; it does not impose a hard bound on all outstanding spend. For raw chat forwarding, `complete` means the provider response envelope was received and stored; the host adapter must still interpret provider status and semantics. Image generation has its own result classification. Original-operation lookup reads stored results, not a provider job queue or a resumable SSE stream. Missing usage can remain unresolved. The current generation adapters support specified text-to-image interfaces. Declaring a video or voice capability does not make it executable without an adapter. Payment collection, exchange rates, refunds, and business authorization remain separate responsibilities. ACC, the Agent Capability Contract, is a separate implementation-neutral project. It describes business capabilities; the billing policy here is a BailingHub implementation choice, not an ACC requirement. If you are adding billing to a local agent, start with the existing tool loop and trace what changes when the gateway is inserted. Every new wait, limit, or retry decision should have a clear owner and a reason. Which boundary is hardest in your system: keeping streaming responsive, bounding unsettled spend, or recovering a dispatched operation without submitting it twice?