Benchmark · measured, not claimed
A live A/B measurement, not a slogan. The same real multi-file task, resumed at a session boundary with and without VeriCommand’s record — every payload tokenized and counted.
Yes — from the first handoff.
The record’s resume read measured 22.6× smaller than re-reading the files an agent touched, and the record costs so little to run (~427 tokens a session) that it turns net-positive the moment work crosses a single session or a second model. In one continuous session with no boundary, it costs a negligible 427 tokens.
01What was measured #
The token question comes down to one thing: what it costs to re-establish working state at a context boundary — a session resume, a hand-off to another model, or a restart after the context window compacts — plus the small overhead the record adds while you work.
The task is real: RatePilot’s team-seats feature, which touched eight files across billing, server, data access, templates, and tests — exactly the kind of multi-session work where hand-off matters. The tokenizer is tiktoken o200k_base
, applied identically to both sides.
Without the record. To resume, the agent re-reads the files it touched to reconstruct what it decided and what’s left. Cost = the tokens of those files.With the record. To resume, the agent makes oneread_truth
call and gets a compact, chain-verified state block — captured live from the record, not estimated — plus the overhead of the governance calls made while working.
02The numbers #
Without the record — the files an agent re-reads to resume:
| File re-read to reconstruct state | Tokens |
|---|---|
| server.py | 11,276 |
| billing.py | 2,427 |
| subscribe.html | 2,368 |
| team.html | 1,731 |
| test_team_seats.py | 1,670 |
| supabase_rest.py | 1,530 |
| test_trial_metering.py | 1,034 |
| test_billing_scope.py | 861 |
| Resume cost / boundary | 22,897 |
With the record — one read_truth | 1,013 |
| Per-session overhead — create + dispatch + 5 emits + submit | 427 |
| Savings per boundary (22,897 − 1,013) | +21,884 |
03The crossover #
Net tokens saved, by how many sessions the work spans. Overhead is paid every session; the ~22K resume saving lands at every boundary between them.
A single continuous session with no hand-off is the only case where the record costs rather than saves — and there it’s 427 tokens.
04The skeptic pass #
“An agent doesn’t re-read everything to resume.” Fair. So we varied the without-record resume down to a lean compaction summary, and varied the record read up to its heaviest form. The number below is the minimum sessions to net-positive. It holds at one hand-off almost everywhere — the overhead is simply too small to move it.
| Without-record resume → | read_truth (1.0K) | raw return (2.5K) | heaviest read (4K) |
|---|---|---|---|
| compaction summary (~3K) | 2 sessions | 7 sessions | never | | 2 key files (~9K) | 2 sessions | 2 sessions | 2 sessions | | server + billing (~13.7K) | 2 sessions | 2 sessions | 2 sessions | | full re-read (22.9K) | 2 sessions | 2 sessions | 2 sessions |
The only “never” is the corner where you’d have resumed from a tiny summary anyway and chose the heaviest read — a wash, not a loss.
05Corroborated by real runs #
VeriCommand’s own flightdeck_burn
tool reads actual run logs. Across 29 real runs it measured 5.65M tokens (13 with reported usage; the rest logged honestly as unknown, never zero). The most expensive single run burned 858,432 tokens across 43 turns — and the tool’s own diagnosis names the exact mechanism this benchmark is about: “re-ran an unnarrowed full suite 7×… every full run returns its whole output into context, and context is re-read each turn, so this compounds.” Re-reading bulky context every turn is precisely the cost a compact, structured hand-off removes.
06Honest limits #
Proxy tokenizer.o200k_base
is not Claude’s exact tokenizer; absolute counts are ±~10–15%. Both sides use it identically, so the ratio and crossover are robust to the choice.The without-record cost is a floor. It counts only file re-reads — not the re-searching, directory listing, and git inspection a real resume also incurs. The true cost is higher, so the savings shown are conservative.Payloads, not a single live run. This measures the token payloads that drive cost, grounded in the real run logs above — not one noisy end-to-end trial.Not magic in one session. With no boundary to amortize over, the record is a small net cost. Its value is at boundaries, and it compounds with every extra session or model.Tokens aren’t the only saving. A verifiable record also stops an agent re-doing or wrong-pathing work it already did — rework that this measurement doesn’t even count. Two measured numbers frame it: one independent review costs532 tokens; one real agent turn is**~32.7K**(median, run logs).Part twodoes that arithmetic, and says where it’s modeled.
07The claim it supports #
Not “VeriCommand saves tokens” — too flat to be true. The measured claim is sharper: the record pays for itself the moment work crosses a session or a model — a resume ~23× smaller, net-positive at the first hand-off, scaling to roughly 190K tokens saved over ten sessions on a mid-size task. Single-shot work in one window doesn’t need it; multi-session and multi-model work is exactly where the math turns in its favor.
Method + data reproducible · tiktoken o200k_base · real RatePilot team-seats task · live record payloads · flightdeck_burn run logs · 2026-09-02
Part two · Pro
Credits burn fastest going the wrong way. What does the review save? #
A different axis from everything above. The free record makes resumes cheap. Pro is the independent, different-vendor review — and a review spends tokens to run. Its payoff is avoided rework: catching drift before the agent burns turns building and debugging the wrong thing.
So the review is a rounding error against a wasted turn. When a catch prevents a wrong-path detour, the return is lopsided:
| Wrong-path detour caught | Tokens saved | Return on the review |
|---|---|---|
| 1 turn — one wrong turn | ~32,100 | 61× |
| 3 turns — build wrong, fail, redo | ~97,400 | 184× |
| 5 turns — deep detour | ~162,800 | 307× |
Worst case for Pro — a heavy 5,000-token full-return review against your cheapest measured turn, catching just one wrong turn — still returns 4× its cost.
Measured vs modeled. The review cost and the per-turn cost are measured — the latter from realflightdeck_burn
logs. The detour length is amodeled scenario, shown as a range, not a claimed fact.** It realizes on a catch.**The saving lands only when the review actually catches a drift. This is not “Pro always saves X” — it’s “when it catches one, the math is lopsided.”A different axis. This is avoided rework, not the cheaper resumes measured above. The two savings stack; they don’t overlap.
The measured claim: a review costs a fraction of one agent turn, and a wrong path is many turns. The most expensive tokens you’ll spend are the ones going the wrong way — which is exactly what an independent review is there to stop.