Build a Budget Variance Ledger for AI API Gateways

A developer has proposed a budget variance ledger format for AI API gateways that compares pre-release cost estimates against gateway receipts using the same dated pricing source and route version approved before shipping. The ledger records token counts, retry attempts, cache evidence, and pricing versions per workload slice without exposing prompts, customer data, or credentials, using AIWave's public pricing and route tables as a worked example. The author argues the artifact helps engineering, finance, and procurement explain model spend variance without reconstructing assumptions after the invoice arrives.

Most AI API budget reviews start too late. The invoice arrives. Someone exports usage. Someone else finds the rate card that was current when the feature launched. A third person asks whether cache hits were counted, whether retries were included, and whether the model ID in the product code is the same one shown in the planning workbook. By then the team is no longer debugging a single request. It is reconstructing a chain of assumptions. A budget variance ledger gives that chain a compact shape. It compares what the release expected to happen with what the gateway receipts say actually happened, using the same dated pricing source and route version that the team approved before shipping. This is not a replacement for billing. It is a review artifact for Tier 1 and Tier 2 engineering teams that need to explain model spend without exposing prompts, customer data, or reusable credentials. I will use AIWave as the concrete example because its public pricing snapshot and live route table can be checked without a private account. As of September 19, 2026, https://aiwave.live/api/v1/pricing returned 56 model rows, currency USD , unit per 1m text tokens , checked=2026-09-10 , updated at=2026-09-18 , and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5 . The live route table at https://aiwave.live/api/pricing returned 68 route rows, success=true , pricing version a42d372ccf0b5dd13ecf71203521f9d2 , and group ratios default=1 and vip=0.9 . Those values are deliberately dated. A variance ledger should never pretend that a price row is timeless. Planning math usually begins with a friendly formula: estimated cost = input tokens input rate + cached input tokens cache hit rate + output tokens output rate That formula is useful, but it hides the parts that change during production: | Source of variance | What usually went missing | |---|---| | Model route changed | The release used a different model ID than the workbook | | Output grew | The product allowed longer answers than the estimate assumed | | Retries multiplied | Timeout or 429 handling repeated the same user action | | Cache did not hold | The forecast used a cache-hit row without receipt proof | | Group multiplier differed | The API key or account group changed the applied rate | | Pricing version moved | The checked source was refreshed after release approval | The ledger does not need to accuse anyone. It just needs to make these differences visible. A row should compare one workload slice, not the whole business. Start with a feature, customer tier, route policy, or scheduled job. { "ledger id": "support-summary-2026-09-19", "checked at": "2026-09-19T13:20:00Z", "feature": "support summary", "client shape": "openai chat completions", "model id": "deepseek-flash", "pricing snapshot": { "url": "https://aiwave.live/api/v1/pricing", "checked": "2026-09-10", "updated at": "2026-09-18", "pricing version": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5" }, "route table": { "url": "https://aiwave.live/api/pricing", "pricing version": "a42d372ccf0b5dd13ecf71203521f9d2", "group ratio": { "default": 1, "vip": 0.9 } }, "expected": { "requests": 1000, "input tokens": 180000000, "cached input tokens": 0, "output tokens": 22000000, "max retries": 1 }, "actual": { "requests": 1000, "input tokens": 176400000, "cached input tokens": 24800000, "output tokens": 31800000, "retry attempts": 86 }, "variance review": { "output tokens": "above plan", "cache evidence": "observed in receipts", "retry policy": "within limit", "verdict": "review output cap before next release" } } There is no prompt text in that object. There is no API key. There is no user identifier. The point is to let engineering, finance, and procurement inspect the spend mechanics without turning the review file into a private data dump. The ledger can stay plain JSON, while a small check computes the variance bands. python from future import annotations from dataclasses import dataclass @dataclass frozen=True class Rate: input: float cached input: float | None output: float def usd per million tokens: int, rate: float - float: return tokens / 1 000 000 rate def estimate row: dict, rate: Rate, group ratio: float = 1.0 - float: expected = row "expected" cached tokens = expected.get "cached input tokens", 0 fresh tokens = max expected "input tokens" - cached tokens, 0 cached rate = rate.cached input if rate.cached input is not None else rate.input subtotal = usd per million fresh tokens, rate.input + usd per million cached tokens, cached rate + usd per million expected "output tokens" , rate.output return subtotal group ratio def actual cost row: dict, rate: Rate, group ratio: float = 1.0 - float: actual = row "actual" cached tokens = actual.get "cached input tokens", 0 fresh tokens = max actual "input tokens" - cached tokens, 0 cached rate = rate.cached input if rate.cached input is not None else rate.input subtotal = usd per million fresh tokens, rate.input + usd per million cached tokens, cached rate + usd per million actual "output tokens" , rate.output return subtotal group ratio For a production gate, do not hard-code the rate values. Pull them from the dated snapshot, match the exact model ID, and fail closed if the pricing version or model row changed. php def classify variance planned: float, observed: float - str: if planned <= 0: return "missing plan" ratio = observed - planned / planned if ratio 0.20: return "review required" if ratio 0.05: return "watch" if ratio < -0.20: return "under plan explain" return "within band" The exact thresholds are a business decision. The important part is that they are explicit before the run, not invented after the invoice. A single percentage is not enough. A useful ledger names the likely cause. { "variance causes": { "field": "output tokens", "expected": 22000000, "actual": 31800000, "action": "tighten answer length or streaming stop rule" }, { "field": "retry attempts", "expected max": 100, "actual": 86, "action": "no change" }, { "field": "pricing version", "expected": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5", "actual": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5", "action": "no change" } } That structure makes review calmer. If output growth caused the variance, the answer is not to debate the entire provider strategy. It is to review the product behavior that let output grow. If pricing version drift caused the variance, the answer is not to blame retries. It is to update the source snapshot and re-approve the route. Cache-hit pricing is easy to misuse because it looks precise in a table. A variance ledger should separate three ideas: If your gateway cannot prove cache status per request, the ledger should not classify reduced spend as cache success. It should say the cause is unknown. That may feel strict, but it protects the next release. Teams should not scale a workload because a spreadsheet predicted cache behavior. They should scale it because receipts show the behavior actually happened. The most useful variance reviews are reproducible. Keep a tiny replay fixture next to the ledger so the team can rerun the same calculation when the route, snapshot, or output cap changes. The fixture should be synthetic. Do not replay customer prompts. Instead, store a narrow set of redacted receipt shapes: { "case": "normal path", "model id": "deepseek-flash", "prompt tokens": 180000, "completion tokens": 22000, "cached input tokens": 0, "retry attempts": 0, "group": "default" }, { "case": "long answer path", "model id": "deepseek-flash", "prompt tokens": 176000, "completion tokens": 42000, "cached input tokens": 18000, "retry attempts": 1, "group": "default" } Run the fixture against the same pricing snapshot that the ledger names. If the code can only pass with today's live table, the review is not reproducible. If it can pass with the pinned snapshot and also show the diff against the live table, the reviewer can separate two questions: That distinction matters. A product bug and a pricing-source update need different owners. Once the ledger classifies variance, route the alert by cause. | Cause | First owner | |---|---| | Output tokens above plan | Product or prompt owner | | Retry attempts above plan | Platform reliability owner | | Model ID mismatch | Release engineering owner | | Cache evidence missing | Gateway instrumentation owner | | Pricing version drift | Pricing-source owner | | Group multiplier mismatch | Account or key-management owner | This keeps the review from becoming a vague "AI costs are high" meeting. The right owner gets a concrete field, a dated source, and a next action. For example, output growth might lead to a stricter response contract, streaming stop rule, or per-feature max token cap. Retry growth might lead to better timeout budgets or idempotency checks. Pricing drift might simply require re-approving the route with a new source date. The ledger should make those paths visible without claiming that any one route is universally better. A good variance ledger is boring in the best way: That makes it safe to attach to a release review, an incident recap, or a procurement answer. Public copy should stay narrower than the ledger. Safe public language might say: The route was reviewed against a dated pricing snapshot and request-level receipts. Risky language would claim a universal cost outcome. The ledger can support operational confidence, but it cannot predict every workload shape, retry pattern, cache behavior, account group, or future price row. The practical question for a gateway team is simple: When spend changes, can we explain whether the cause was model choice, output length, retry behavior, cache evidence, group multiplier, or pricing version drift? If the answer is yes, the budget review stops being archaeology.