Most AI API budget reviews start too late.
The invoice arrives. Someone exports usage. Someone else finds the rate card that was current when the feature launched. A third person asks whether cache hits were counted, whether retries were included, and whether the model ID in the product code is the same one shown in the planning workbook.
By then the team is no longer debugging a single request. It is reconstructing a chain of assumptions.
A budget variance ledger gives that chain a compact shape. It compares what the release expected to happen with what the gateway receipts say actually happened, using the same dated pricing source and route version that the team approved before shipping.
This is not a replacement for billing. It is a review artifact for Tier 1 and Tier 2 engineering teams that need to explain model spend without exposing prompts, customer data, or reusable credentials.
I will use AIWave as the concrete example because its public pricing snapshot and live route table can be checked without a private account. As of September 19, 2026, https://aiwave.live/api/v1/pricing returned 56 model rows, currency USD, unit per_1m_text_tokens, checked=2026-09-10, updated_at=2026-09-18, and pricing version 83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5. The live route table at https://aiwave.live/api/pricing returned 68 route rows, success=true, pricing version a42d372ccf0b5dd13ecf71203521f9d2, and group ratios default=1 and vip=0.9.
Those values are deliberately dated. A variance ledger should never pretend that a price row is timeless.
Planning math usually begins with a friendly formula:
estimated_cost =
input_tokens * input_rate
+ cached_input_tokens * cache_hit_rate
+ output_tokens * output_rate
That formula is useful, but it hides the parts that change during production:
| Source of variance | What usually went missing |
|---|---|
| Model route changed | The release used a different model ID than the workbook |
| Output grew | The product allowed longer answers than the estimate assumed |
| Retries multiplied | Timeout or 429 handling repeated the same user action |
| Cache did not hold | The forecast used a cache-hit row without receipt proof |
| Group multiplier differed | The API key or account group changed the applied rate |
| Pricing version moved | The checked source was refreshed after release approval |
The ledger does not need to accuse anyone. It just needs to make these differences visible.
A row should compare one workload slice, not the whole business. Start with a feature, customer tier, route policy, or scheduled job.
{
"ledger_id": "support-summary-2026-09-19",
"checked_at": "2026-09-19T13:20:00Z",
"feature": "support_summary",
"client_shape": "openai_chat_completions",
"model_id": "deepseek-flash",
"pricing_snapshot": {
"url": "https://aiwave.live/api/v1/pricing",
"checked": "2026-09-10",
"updated_at": "2026-09-18",
"pricing_version": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5"
},
"route_table": {
"url": "https://aiwave.live/api/pricing",
"pricing_version": "a42d372ccf0b5dd13ecf71203521f9d2",
"group_ratio": {
"default": 1,
"vip": 0.9
}
},
"expected": {
"requests": 1000,
"input_tokens": 180000000,
"cached_input_tokens": 0,
"output_tokens": 22000000,
"max_retries": 1
},
"actual": {
"requests": 1000,
"input_tokens": 176400000,
"cached_input_tokens": 24800000,
"output_tokens": 31800000,
"retry_attempts": 86
},
"variance_review": {
"output_tokens": "above_plan",
"cache_evidence": "observed_in_receipts",
"retry_policy": "within_limit",
"verdict": "review_output_cap_before_next_release"
}
}
There is no prompt text in that object. There is no API key. There is no user identifier. The point is to let engineering, finance, and procurement inspect the spend mechanics without turning the review file into a private data dump.
The ledger can stay plain JSON, while a small check computes the variance bands.
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True)
class Rate:
input: float
cached_input: float | None
output: float
def usd_per_million(tokens: int, rate: float) -> float:
return tokens / 1_000_000 * rate
def estimate(row: dict, rate: Rate, group_ratio: float = 1.0) -> float:
expected = row["expected"]
cached_tokens = expected.get("cached_input_tokens", 0)
fresh_tokens = max(expected["input_tokens"] - cached_tokens, 0)
cached_rate = rate.cached_input if rate.cached_input is not None else rate.input
subtotal = (
usd_per_million(fresh_tokens, rate.input)
+ usd_per_million(cached_tokens, cached_rate)
+ usd_per_million(expected["output_tokens"], rate.output)
)
return subtotal * group_ratio
def actual_cost(row: dict, rate: Rate, group_ratio: float = 1.0) -> float:
actual = row["actual"]
cached_tokens = actual.get("cached_input_tokens", 0)
fresh_tokens = max(actual["input_tokens"] - cached_tokens, 0)
cached_rate = rate.cached_input if rate.cached_input is not None else rate.input
subtotal = (
usd_per_million(fresh_tokens, rate.input)
+ usd_per_million(cached_tokens, cached_rate)
+ usd_per_million(actual["output_tokens"], rate.output)
)
return subtotal * group_ratio
For a production gate, do not hard-code the rate values. Pull them from the dated snapshot, match the exact model ID, and fail closed if the pricing version or model row changed.
def classify_variance(planned: float, observed: float) -> str:
if planned <= 0:
return "missing_plan"
ratio = (observed - planned) / planned
if ratio > 0.20:
return "review_required"
if ratio > 0.05:
return "watch"
if ratio < -0.20:
return "under_plan_explain"
return "within_band"
The exact thresholds are a business decision. The important part is that they are explicit before the run, not invented after the invoice.
A single percentage is not enough. A useful ledger names the likely cause.
{
"variance_causes": [
{
"field": "output_tokens",
"expected": 22000000,
"actual": 31800000,
"action": "tighten_answer_length_or_streaming_stop_rule"
},
{
"field": "retry_attempts",
"expected_max": 100,
"actual": 86,
"action": "no_change"
},
{
"field": "pricing_version",
"expected": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5",
"actual": "83f77abde81ee3a096a672ed959ccc096f5d37a45c177ae8e03229456b5415a5",
"action": "no_change"
}
]
}
That structure makes review calmer. If output growth caused the variance, the answer is not to debate the entire provider strategy. It is to review the product behavior that let output grow.
If pricing version drift caused the variance, the answer is not to blame retries. It is to update the source snapshot and re-approve the route.
Cache-hit pricing is easy to misuse because it looks precise in a table. A variance ledger should separate three ideas:
If your gateway cannot prove cache status per request, the ledger should not classify reduced spend as cache success. It should say the cause is unknown.
That may feel strict, but it protects the next release. Teams should not scale a workload because a spreadsheet predicted cache behavior. They should scale it because receipts show the behavior actually happened.
The most useful variance reviews are reproducible. Keep a tiny replay fixture next to the ledger so the team can rerun the same calculation when the route, snapshot, or output cap changes.
The fixture should be synthetic. Do not replay customer prompts. Instead, store a narrow set of redacted receipt shapes:
[
{
"case": "normal_path",
"model_id": "deepseek-flash",
"prompt_tokens": 180000,
"completion_tokens": 22000,
"cached_input_tokens": 0,
"retry_attempts": 0,
"group": "default"
},
{
"case": "long_answer_path",
"model_id": "deepseek-flash",
"prompt_tokens": 176000,
"completion_tokens": 42000,
"cached_input_tokens": 18000,
"retry_attempts": 1,
"group": "default"
}
]
Run the fixture against the same pricing snapshot that the ledger names. If the code can only pass with today's live table, the review is not reproducible. If it can pass with the pinned snapshot and also show the diff against the live table, the reviewer can separate two questions:
That distinction matters. A product bug and a pricing-source update need different owners.
Once the ledger classifies variance, route the alert by cause.
| Cause | First owner |
|---|---|
| Output tokens above plan | Product or prompt owner |
| Retry attempts above plan | Platform reliability owner |
| Model ID mismatch | Release engineering owner |
| Cache evidence missing | Gateway instrumentation owner |
| Pricing version drift | Pricing-source owner |
| Group multiplier mismatch | Account or key-management owner |
This keeps the review from becoming a vague "AI costs are high" meeting. The right owner gets a concrete field, a dated source, and a next action.
For example, output growth might lead to a stricter response contract, streaming stop rule, or per-feature max token cap. Retry growth might lead to better timeout budgets or idempotency checks. Pricing drift might simply require re-approving the route with a new source date.
The ledger should make those paths visible without claiming that any one route is universally better.
A good variance ledger is boring in the best way:
That makes it safe to attach to a release review, an incident recap, or a procurement answer.
Public copy should stay narrower than the ledger. Safe public language might say:
The route was reviewed against a dated pricing snapshot and request-level receipts.
Risky language would claim a universal cost outcome. The ledger can support operational confidence, but it cannot predict every workload shape, retry pattern, cache behavior, account group, or future price row.
The practical question for a gateway team is simple:
When spend changes, can we explain whether the cause was model choice, output length, retry behavior, cache evidence, group multiplier, or pricing version drift?
If the answer is yes, the budget review stops being archaeology.