{"slug": "show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013", "title": "Show HN: I measured resuming an AI coding session: 22,897 tokens vs. 1,013", "summary": "VeriCommand's benchmark shows that resuming an AI coding session with its record uses 1,013 tokens versus 22,897 tokens without it, a 22.6× reduction, saving 21,884 tokens per session boundary. The measurement, based on a real multi-file task touching eight files, found the record costs only 427 tokens per session and turns net-positive after a single hand-off. Across 29 real runs, VeriCommand's flightdeck_burn tool measured 5.65M tokens burned, with the most expensive run consuming 858,432 tokens across 43 turns.", "body_md": "Benchmark · measured, not claimed\n\n# Does keeping the work on one record actually save tokens?\n\nA live A/B measurement, not a slogan. The same real multi-file task, resumed at a session boundary with and without VeriCommand’s record — every payload tokenized and counted.\n\n**Yes — from the first handoff.**\n\nThe record’s resume read measured **22.6× smaller** than re-reading the files an agent touched, and the record costs so little to run (~427 tokens a session) that it turns net-positive the moment work crosses a single session or a second model. In one continuous session with no boundary, it costs a negligible 427 tokens.\n\n## 01What was measured\n\nThe token question comes down to one thing: what it costs to re-establish working state at a context boundary — a session resume, a hand-off to another model, or a restart after the context window compacts — plus the small overhead the record adds while you work.\n\nThe task is real: RatePilot’s team-seats feature, which touched eight files across billing, server, data access, templates, and tests — exactly the kind of multi-session work where hand-off matters. The tokenizer is `tiktoken o200k_base`\n\n, applied identically to both sides.\n\n**Without the record.** To resume, the agent re-reads the files it touched to reconstruct what it decided and what’s left. Cost = the tokens of those files.**With the record.** To resume, the agent makes one`read_truth`\n\ncall and gets a compact, chain-verified state block — captured live from the record, not estimated — plus the overhead of the governance calls made while working.\n\n## 02The numbers\n\nWithout the record — the files an agent re-reads to resume:\n\n| File re-read to reconstruct state | Tokens |\n|---|---|\n| server.py | 11,276 |\n| billing.py | 2,427 |\n| subscribe.html | 2,368 |\n| team.html | 1,731 |\n| test_team_seats.py | 1,670 |\n| supabase_rest.py | 1,530 |\n| test_trial_metering.py | 1,034 |\n| test_billing_scope.py | 861 |\n| Resume cost / boundary | 22,897 |\n\nWith the record — one `read_truth` | 1,013 |\n| Per-session overhead — create + dispatch + 5 emits + submit | 427 |\n| Savings per boundary (22,897 − 1,013) | +21,884 |\n\n## 03The crossover\n\nNet tokens saved, by how many sessions the work spans. Overhead is paid every session; the ~22K resume saving lands at every boundary between them.\n\nA single continuous session with no hand-off is the only case where the record costs rather than saves — and there it’s 427 tokens.\n\n## 04The skeptic pass\n\n“An agent doesn’t re-read *everything* to resume.” Fair. So we varied the without-record resume down to a lean compaction summary, and varied the record read up to its heaviest form. The number below is the minimum sessions to net-positive. It holds at one hand-off almost everywhere — the overhead is simply too small to move it.\n\n| Without-record resume → | read_truth (1.0K) | raw return (2.5K) | heaviest read (4K) |\n|---|---|---|---|\n| compaction summary (~3K) | 2 sessions | 7 sessions | never |\n| 2 key files (~9K) | 2 sessions | 2 sessions | 2 sessions |\n| server + billing (~13.7K) | 2 sessions | 2 sessions | 2 sessions |\n| full re-read (22.9K) | 2 sessions | 2 sessions | 2 sessions |\n\nThe only “never” is the corner where you’d have resumed from a tiny summary anyway *and* chose the heaviest read — a wash, not a loss.\n\n## 05Corroborated by real runs\n\nVeriCommand’s own `flightdeck_burn`\n\ntool reads actual run logs. Across 29 real runs it measured 5.65M tokens (13 with reported usage; the rest logged honestly as unknown, never zero). The most expensive single run burned **858,432 tokens across 43 turns** — and the tool’s own diagnosis names the exact mechanism this benchmark is about: *“re-ran an unnarrowed full suite 7×… every full run returns its whole output into context, and context is re-read each turn, so this compounds.”* Re-reading bulky context every turn is precisely the cost a compact, structured hand-off removes.\n\n## 06Honest limits\n\n**Proxy tokenizer.**`o200k_base`\n\nis not Claude’s exact tokenizer; absolute counts are ±~10–15%. Both sides use it identically, so the ratio and crossover are robust to the choice.**The without-record cost is a floor.** It counts only file re-reads — not the re-searching, directory listing, and git inspection a real resume also incurs. The true cost is higher, so the savings shown are conservative.**Payloads, not a single live run.** This measures the token payloads that drive cost, grounded in the real run logs above — not one noisy end-to-end trial.**Not magic in one session.** With no boundary to amortize over, the record is a small net cost. Its value is at boundaries, and it compounds with every extra session or model.**Tokens aren’t the only saving.** A verifiable record also stops an agent re-doing or wrong-pathing work it already did — rework that this measurement doesn’t even count. Two measured numbers frame it: one independent review costs**532 tokens**; one real agent turn is**~32.7K**(median, run logs).[Part two](#pro)does that arithmetic, and says where it’s modeled.\n\n## 07The claim it supports\n\nNot “VeriCommand saves tokens” — too flat to be true. The measured claim is sharper: **the record pays for itself the moment work crosses a session or a model** — a resume ~23× smaller, net-positive at the first hand-off, scaling to roughly 190K tokens saved over ten sessions on a mid-size task. Single-shot work in one window doesn’t need it; multi-session and multi-model work is exactly where the math turns in its favor.\n\nMethod + data reproducible · tiktoken o200k_base · real RatePilot team-seats task · live record payloads · flightdeck_burn run logs · 2026-09-02\n\nPart two · Pro\n\n## Credits burn fastest going the wrong way. What does the review save?\n\nA different axis from everything above. The free record makes **resumes** cheap. Pro is the independent, different-vendor review — and a review *spends* tokens to run. Its payoff is **avoided rework**: catching drift before the agent burns turns building and debugging the wrong thing.\n\nSo the review is a rounding error against a wasted turn. When a catch prevents a wrong-path detour, the return is lopsided:\n\n| Wrong-path detour caught | Tokens saved | Return on the review |\n|---|---|---|\n| 1 turn — one wrong turn | ~32,100 | 61× |\n| 3 turns — build wrong, fail, redo | ~97,400 | 184× |\n| 5 turns — deep detour | ~162,800 | 307× |\n\nWorst case for Pro — a heavy 5,000-token full-return review against your *cheapest* measured turn, catching just one wrong turn — still returns 4× its cost.\n\n**Measured vs modeled.** The review cost and the per-turn cost are measured — the latter from real`flightdeck_burn`\n\nlogs. The detour length is a**modeled scenario**, shown as a range, not a claimed fact.** It realizes on a catch.**The saving lands only when the review actually catches a drift. This is not “Pro always saves X” — it’s “when it catches one, the math is lopsided.”**A different axis.** This is avoided rework, not the cheaper resumes measured above. The two savings stack; they don’t overlap.\n\nThe measured claim: **a review costs a fraction of one agent turn, and a wrong path is many turns.** The most expensive tokens you’ll spend are the ones going the wrong way — which is exactly what an independent review is there to stop.", "url": "https://wpnews.pro/news/show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013", "canonical_source": "https://vericommand.net/benchmark", "published_at": "2026-09-03 13:00:37+00:00", "updated_at": "2026-09-03 13:22:59.698460+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "large-language-models"], "entities": ["VeriCommand", "RatePilot", "flightdeck_burn"], "alternates": {"html": "https://wpnews.pro/news/show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013", "markdown": "https://wpnews.pro/news/show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013.md", "text": "https://wpnews.pro/news/show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013.txt", "jsonld": "https://wpnews.pro/news/show-hn-i-measured-resuming-an-ai-coding-session-22897-tokens-vs-1013.jsonld"}}