Wattage: A token-spend profiler and cost-regression gate for AI agents

Wattage, an open-source token-spend profiler and cost-regression gate for AI agents, analyzes OpenTelemetry traces to identify token waste, price inefficiencies in real dollars, and prescribe fixes, including failing CI when changes increase costs. Its convergence engine detects non-convergent agent loops with 1.00 F1 score versus 0.25 for exact-match baselines, and a prefix_churn fix simulation shows 44.7% cost reduction on a real trace. The tool is fully offline, requires no config or API key, and outputs terminal reports or HTML flame graphs.

A Kill-A-Watt meter for your AI agents. Point it at a trace and it tells you exactly where your tokens are being burned and wasted, prices each waste pattern in real dollars, prescribes a fix, and can fail your CI when a change makes your agent measurably more expensive. A real captured agent trace see provenance — Wattage catches a stable prompt prefix being re-sent instead of cached, prices the waste, and prescribes the fix. Regenerate this GIF with vhs docs/assets/demo.tape see the tape file for the exact command . uvx wattage report trace.json No config file, no API key, fully offline — point it at an OTLP JSON https://opentelemetry.io/docs/specs/otlp/ trace export and it prices every call and runs every detector. Don't have a trace yet? Getting your first trace /faizannraza/wattage/blob/main/docs/getting-started.md covers both "I already have OTel traces" and "I have zero instrumentation" a runnable, 5-minute path from nothing to a real, priced report . Or try it right now against the fixture shipped in this repo: git clone https://github.com/faizannraza/wattage cd wattage && uv sync uv run wattage report examples/sample trace.json ╭──── ⚡ wattage — examples/sample trace.json ────╮ │ Token Efficiency: A 100 Total cost: $0.0602 │ │ quality: unmeasured │ ╰─────────────────────────────────────────────────╯ Token breakdown ┏━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┃ Category ┃ Tokens ┃ ┡━━━━━━━━━━━━━━━━╇━━━━━━━━┩ │ input │ 18450 │ │ output │ 320 │ │ cache read │ 0 │ │ cache creation │ 0 │ │ reasoning │ 0 │ └────────────────┴────────┘ No findings — this trace looks efficient. pricing: 2026-07-18-verified Or get a self-contained, shareable HTML flame graph instead of the terminal view: uv run wattage report examples/sample trace.json --html report.html Wattage's standout feature is the convergence engine — the nonconvergence detector, which catches an agent thrashing through a loop without making real progress, including patterns a naive exact-match duplicate detector structurally cannot see a retry with a fresh timestamp each time, an oscillation between two strategies, a "productive-looking" stall where every call is technically unique but nothing is actually learned . Rather than assert that, we built a hand-reviewed set of 10 labeled synthetic loops and benchmarked Wattage's classifier against a real SHA-256 exact-match baseline implementation: | Classifier | Precision | Recall | F1 | |---|---|---|---| Wattage | 1.00 | 1.00 | 1.00 | | SHA-256 exact-match | 1.00 | 0.14 | 0.25 | Reproduce it yourself — no cherry-picking, no hidden setup: uv run python -m benchmarks.harness And on a genuine captured agent trace not synthetic — see benchmarks/traces/README.md /faizannraza/wattage/blob/main/benchmarks/traces/README.md for provenance , Wattage's prefix churn fix simulation shows a 44.7% cost reduction $0.000199 → $0.000110 from enabling prompt caching on the stable prefix — small dollar figures because it's a 3-turn demo trace, but the mechanism is identical at production scale. Run it against your own traces for numbers that matter: python uv run python -c "from benchmarks.frontier import build frontier; print build frontier " Full methodology: The Convergence Engine /faizannraza/wattage/blob/main/docs/convergence.md . uv run wattage badge trace.json --out wattage-badge.svg Wattage wattage-badge.svg Wire --badge-out into your CI job see below so it regenerates on every merge to your default branch, and the badge in your README stays live. Three surfaces, one normalized data model underneath sessions → tasks → loops → iterations → calls , built from OpenTelemetry GenAI semantic-convention https://opentelemetry.io/docs/specs/semconv/gen-ai/ traces: - — ingests a trace, prices every call against a vendored, dated pricing snapshot, and runs eight detectors: wattage report Detector Catches prefix churn Stable context re-sent instead of cached cache gap Caching attempted but under-redeemed by later reads verbosity Output far beyond what the step needed redundant tool calls The same tool call repeated exact or fuzzy nonconvergence Loops that thrash, oscillate, or stall without progress retrieval thrash Repeated retrieval that never yields relevant results model mismatch A pricier model doing work a cheaper one could handle reasoning overspend Heavy reasoning-token spend on a simple step Every finding is priced in real dollars, includes a concrete fix, and is tagged with a quality risk tier none / low / review — a fix that could plausibly change output quality a model downgrade, less reasoning only counts toward your score once a --quality map backs it with real evidence. Full detail: Detectors /faizannraza/wattage/blob/main/docs/detectors/index.md . - — a single 0–100 Token Efficiency grade for a README badge or a CI gate. wattage score / wattage badge - — the cost-regression gate below . wattage ci Wattage never fabricates a number: an unpriced model leaves that call's cost at zero and fails wattage ci loudly, exit code 4 rather than guessing; an unmeasured quality signal is reported as unmeasured , not assumed fine. .github/workflows/wattage.yml name: Wattage on: pull request: paths: "agents/ ", "prompts/ ", "src/ " concurrency: group: wattage-${{ github.ref }} cancel-in-progress: true jobs: token-efficiency: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Generate trace fixture run: python scripts/run agent fixture.py trace.json - name: Wattage cost-regression gate uses: faizannraza/wattage/action@v0.1.0 with: source: trace.json baseline: .wattage/baseline.json fail-on: "score below:80,cost delta pct above:5,any critical:true" pr-comment: "true" Fails the build exit code 1 when your agent regresses past the threshold you set, posts a per-detector delta table as a PR comment, and emits SARIF shows up in GitHub's Security tab and JUnit XML for any other CI system. The baseline is a small committed JSON file — noise-floor protection is structural, not statistical: it only ever updates on a run that actually passed the gate. This is only half the setup. A PR job runs on a throwaway checkout, so it can't be the thing that updates .wattage/baseline.json on disk — that update needs a second workflow, triggered on push to your default branch, that commits the refreshed baseline and badge back after each merge. Skipping it means every PR compares against the same stale baseline forever. Full reference, with both workflows: CI Integration /faizannraza/wattage/blob/main/docs/ci.md . Detectors are discovered through a Python entry-point group, so adding one doesn't require touching this repo's core pipeline — see CONTRIBUTING.md /faizannraza/wattage/blob/main/CONTRIBUTING.md for the full "write a detector" walkthrough, using cache gap /faizannraza/wattage/blob/main/src/wattage/detectors/cache gap.py as the reference example.