Add Model Spend Behind a Budget Row, Not a Browser Agent

A developer argues that model calls capable of mutating stored state should be gated by a tenant-scoped budget row rather than executed from a browser agent holding a provider key. The proposed reference implementation adds inference_budgets and inference_spend_events tables with a unique idempotency key, so handlers return HTTP 402 and write nothing when a locked budget row cannot cover a conservative token estimate. The author contends the failure point is not the model but the absence of a constraint at the layer that can refuse the write.

The support console fired a rewrite request at 4:11 p.m., and the widget treated a model response like a database grant. The browser held a provider key in local storage, posted a completion, then PATCHed the invoice with whatever paragraph came back. Did anyone ask whether that tenant still had inference budget for ticket rewrites this month? Did a second tab retry the same click and persist a second, slightly different version of the legal text? I think the fashionable answer this week is wrong, and I will not hedge it. An agent that never leaves the browser is a demo of latency, not a delivery path for customer writes. The first layer that fails is not the model, because the model did return usable prose. The missing budget row should have refused the write, and nobody on the path asked it. Free inference is a prototype subsidy, and it is not permission to mutate invoices, comments, or stored schema. If your UI can call a model and your API can persist the result, you already gave write authority to a meter you do not own. Why do we keep wiring it the other way around, week after week, in otherwise serious codebases? The happy path looks cheap until a retry doubles spend and the bad paragraph is already in Postgres. Can you roll that write back from a chat transcript after the toast has already gone green? I cannot, and I do not believe a streaming DOM node will save you either. The contract I want is boring on purpose, which is usually a sign it will survive contact with production. Every model call that can affect stored state must present a tenant, a purpose, and a budget row. If the row cannot cover the call, the handler returns 402 and nothing is written. Walk the request with me as a vertical slice, not as an architecture poster on a wiki. A human clicks rewrite, and the browser should post a command with ticket id , intent , and an idempotency key. That client should not hold a provider key, and it should not pick a model name from a dropdown somebody added during a hack day. Where does this fail first when the spend envelope is not a real row? It fails right after the model returns text, because nothing in the path is allowed to say no. Auth said the user could edit the ticket, and the model said here is a paragraph, so Postgres returned 200. The budget said nothing at all, because it was never asked as a locked row. That is not an artificial intelligence problem hiding in a vendor SDK. That is a missing constraint at the only layer that can refuse a write. I am labeling the next blocks as a reference implementation you can run locally, not as a souvenir from a named cluster. You should keep that label in your own repo until the 402 path is proven with a real status code. The unique constraint on the idempotency key is the whole point of the second table. Network retries must not mint a second spend event and a second invoice rewrite. CREATE TABLE inference budgets id uuid PRIMARY KEY, tenant id uuid NOT NULL, purpose text NOT NULL, period start timestamptz NOT NULL, period end timestamptz NOT NULL, token limit integer NOT NULL, token spent integer NOT NULL DEFAULT 0, UNIQUE tenant id, purpose, period start ; CREATE TABLE inference spend events id uuid PRIMARY KEY, budget id uuid NOT NULL REFERENCES inference budgets id , idempotency key text NOT NULL UNIQUE, provider text NOT NULL, input tokens integer NOT NULL, output tokens integer NOT NULL, status text NOT NULL, created at timestamptz NOT NULL DEFAULT now ; Have you watched a mobile client replay a POST because the toast never rendered on a flaky radio? The second write is almost always the one that lands in support later. A FastAPI handler can refuse the business write when a locked budget row cannot cover a conservative estimate. This sketch estimates input tokens plus a hard output cap, then reconciles after the provider responds, which is the actual seam. Reference implementation: prove the 402 locally. Not production gospel. from fastapi import APIRouter, Depends, HTTPException from pydantic import BaseModel, Field router = APIRouter ESTIMATED OUTPUT CAP = 800 class RewriteCommand BaseModel : ticket id: str intent: str = Field pattern="^rewrite friendly$" idempotency key: str tenant id: str @router.post "/tickets/{ticket id}/rewrite" def rewrite ticket ticket id: str, cmd: RewriteCommand, db=Depends get db , models=Depends get model client , : if cmd.ticket id = ticket id: raise HTTPException 409, "ticket id mismatch" existing = db.fetch spend event cmd.idempotency key if existing: return {"status": existing.status, "spend id": existing.id, "replayed": True} budget = db.lock budget cmd.tenant id, purpose="ticket rewrite" if budget is None: raise HTTPException 403, "no budget row for purpose" estimate = estimate input tokens ticket id + ESTIMATED OUTPUT CAP if budget.token spent + estimate budget.token limit: raise HTTPException 402, "inference budget exhausted" reservation id = db.reserve tokens budget.id, estimate, cmd.idempotency key try: result = models.complete purpose="ticket rewrite", prompt=load ticket prompt ticket id , max tokens=ESTIMATED OUTPUT CAP, except ProviderError as exc: db.release reservation reservation id, reason=str exc raise HTTPException 502, "provider failed; budget released" from exc db.finalize spend reservation id, provider=result.provider, input tokens=result.input tokens, output tokens=result.output tokens, db.apply ticket rewrite ticket id, result.text, spend id=reservation id return {"status": "applied", "spend id": reservation id} Notice what the handler refuses to do, even when the process is running on a scratch box you stood up for a demo. It does not accept a model name from the client, and it does not stream tokens into a websocket that also applies SQL. The purpose string is the product contract, not a log field you add later for analytics. Why would playground chat and ticket rewrite share one pool when they do not share one blast radius? Disclosure: This article was prepared as part of MonkeyCode's product outreach. A scratch environment is the right place to teach the ledger to say no, because that experiment should not ride a paid provider bill. MonkeyCode's free model access and free server option can host that slice while you are still proving the refusal path. Treat the open-source project as a sandbox for the contract, not as a substitute for the contract. If you cannot get a 402 from the free box, you will not get a real refusal in production either. python -m venv .venv source .venv/bin/activate pip install fastapi uvicorn "psycopg binary " uvicorn app:app --reload --port 8080 psql "$DATABASE URL" -c "insert into inference budgets id, tenant id, purpose, period start, period end, token limit values '00000000-0000-0000-0000-000000000001', '11111111-1111-1111-1111-111111111111', 'ticket rewrite', now , now + interval '30 days', 200 ;" curl -sS -X POST "$API/tickets/T-42/rewrite" \ -H "content-type: application/json" \ -H "authorization: Bearer $DEV TOKEN" \ -d '{"ticket id":"T-42","intent":"rewrite friendly", "idempotency key":"rewrite-T-42-01", "tenant id":"11111111-1111-1111-1111-111111111111"}' The first call may apply if the estimate still fits inside that tiny envelope. The second call, with a new idempotency key, should return 402 and leave the ticket body untouched. Replaying the same key should return the original status and still not double-write the invoice. If your status codes do not split that way, the seam is theater, not a contract. I also want a test that the browser never saw a provider key in headers or payloads. That is the difference between a tab-resident agent and a feature that survives a stolen laptop. The test is ugly on purpose, because pretty tests tend to skip the overdraw and only assert that JSON looks friendly. python def test rewrite does not overdraw or write client, db : db.set token limit tenant="t1", purpose="ticket rewrite", limit=50 before = db.ticket body "T-42" first = client.post "/tickets/T-42/rewrite", json=command "T-42", "k1" assert first.status code in {200, 402} second = client.post "/tickets/T-42/rewrite", json=command "T-42", "k2" assert second.status code == 402 if first.status code == 402: assert db.ticket body "T-42" == before assert "sk-" not in str second.headers replay = client.post "/tickets/T-42/rewrite", json=command "T-42", "k1" assert replay.json .get "replayed" in {True, None} or replay.status code == 402 Is that test going to impress a design critique on a clean-code thread? No, and I do not need it to. Spend is a row, writes are a later row, and the model is a vendor behind a purpose. The browser is a client, not a wallet, and it should not get a vote on either row. The failure mode looks like this, and it is more common than a clever browser agent demo. A handler temporarily reads max tokens from the client, and the client sends a larger cap after the first 402. A streaming UI applies a partial sentence when the socket dies at eighty percent, then applies again on resume. A free-server compose file lands in production with the budget check commented out because it made the pitch deck jitter. The model layer looks healthy, and the persist layer looks healthy, so the dashboard stays green. The handoff is an if statement somebody deleted on a Friday so the demo would not blink. Does that sound like a model-quality incident to you? It is a permissions incident with extra tokens attached. Performance testing of the API does not save you if every scenario still has tokens left in the envelope. A load test that never exhausts budget is a load test of the happy path, which you already knew worked. Can you show me the 402 p95, and can you show that ticket bodies did not change on those requests? If you cannot, you have not tested the contract you claim to have. If your product is a local playground with no persistence, a budget row is ceremony, and you should not add it for the aesthetic. If you cannot name a tenant, a purpose, and a period, you are not ready to attach a ledger. A free model will only hide that gap until the first shared demo key leaks into a write path. Also skip this sketch if you need hard multi-region accounting today, because it assumes one database and a row lock. That starting contract is not a billing platform, and I will not pretend otherwise. People who only need a read-only eval slice should stay on the read path until a human grants write authority. Do not use this article as cover for giving a chat widget schema rights. The whole point is the opposite move: prove refusal before you celebrate the prose. Prove the click, the 402, the replay, and the absent provider key before you ship the rewrite. Keep model names on the server, keep purposes distinct, and keep the free server in the eval path until the ledger refuses correctly. Which layer handoff is least stable in your stack right now, budget to model or model to persist? Send a failure state or a response code, not a model name, because I care whether the write happened after the budget already said no.