# An append-only audit log caught two accounting bugs in a 216-star usage tracker > Source: > Published: 2026-07-25 02:17:08+00:00 Last month I wrote about [auditing 34 days of multi-model Claude Code usage](https://dev.to/lizhuojunx86/a-missing-model-line-was-half-my-ai-agent-overspend-auditing-34-days-of-multi-model-claude-code-mc7) and finding that a single missing model line was half my overspend. The core of that piece was a boring engineering decision: ingest every assistant message from Claude Code's session files into an **append-only log keyed by message.id**, and never let the source rewrite history. This is the follow-up I didn't plan to write. The same log just caught two accounting bugs in [splitrail](https://github.com/Piebald-AI/splitrail), a 216-star Rust usage tracker for agentic CLIs — and within twelve days of the first report, both were fixed in an official release: one through my PR, one through a maintainer-authored patch that adopted the report's analysis. Here's the play-by-play, with the numbers. Claude Code rewrites session JSONL files in place when you resume or compact a conversation. Between two scans of the *same file*, five of my assistant messages vanished — usage that was already incurred, gone from the record. Any tracker that recomputes totals from live files inherits this drift: yesterday's numbers change while you sleep. I filed it as [#200](https://github.com/Piebald-AI/splitrail/issues/200). The maintainer confirmed the mechanism the same day — splitrail re-read live files, so messages removed by resume/compaction disappeared from historical totals — and shipped a fix in 3.6.0: a local SQLite history store that persists normalized usage and merges it with current session data before deduplication. Fast, clean work. One wrinkle: he suggested validating by comparing 3.5.9 vs 3.6.0 totals, expecting "3.6.0 should be higher." That's subtly wrong, and the distinction matters for anyone testing this class of fix: **on a cold start the two versions are equal** — the history store has nothing to restore yet. The divergence only appears *across* drift events. So I built the test to create one. Everything ran against a frozen snapshot (APFS clone) of `~/.claude/projects` , so both versions scanned identical bytes, with every invocation under an isolated `$HOME` — own config, own history store, no upload path. Then: | # | assertion | result | |---|---|---| | A | cold-start parity: fresh 3.5.9 == fresh 3.6.0 on identical input | ✅ identical | | B | 3.5.9 across a simulated resume/compact rewrite | ✅ drops by exactly the removed usage | | C | 3.6.0 across the same rewrite | ✅ totals unchanged — history store restores | | D | 3.6.0 restart stability, 3 consecutive runs | ✅ byte-identical output | The simulated rewrite removes the last five assistant message-groups from the largest transcript — the same shape as the original drift event. The whole thing is now a self-contained regression fixture ([PR #208](https://github.com/Piebald-AI/splitrail/pull/208), since merged): it fails red on 3.5.9 and passes green on 3.6.0, which is exactly what you want a regression test to do. The part I care most about: on the transcripts splitrail scans, 3.6.0 agreed with my append-only log **token-exact — 18,548,947 output tokens on both sides** across 13.5k messages. Two independent implementations, different languages, different dedup strategies, same number to the digit. (The tiny message-count delta was zero-usage api-error records splitrail intentionally skips.) When two systems reconcile exactly, every remaining discrepancy is a *finding*, not noise. Because the log records where every message came from, I could classify all of them against the live tree: | class | files | messages | output tokens | |---|---|---|---| | live, main transcript (what splitrail scans) | 76 | 13,704 | 18,548,947 | live, `subagents/**` transcript | 1,423 | 16,160 | 13,738,324 | | vanished (file exists, message.id gone) | 0 | 0 | 0 | | deleted (session file gone) | 874 | 11,821 | 14,238,006 | Row two is the second bug. Claude Code writes subagent transcripts (Task tool: Explore, general-purpose, custom agents) under `projects///subagents/` , at directory depth ≥ 4. Splitrail's discovery hard-caps at depth 2 in three places (`WalkDir…min_depth(2).max_depth(2)` , a `components() == 2` path check, and the glob pattern). Those files are structurally invisible — **54% of my live messages, and roughly a third of the dollars, never entered any total**. The model mix makes the blind spot vivid: splitrail saw 22 of my 5,059 Sonnet messages and 0 of 1,566 Haiku messages. Those models run almost exclusively inside subagents. If you delegate heavily to cheap models — which is exactly what cost-conscious agent users do — your tracker undercounts the most, on the workflows you optimized hardest. Filed as [#207](https://github.com/Piebald-AI/splitrail/issues/207) with the layout, a two-line repro, and a suggested fix. Two days later it was closed — and not by my patch. The maintainer wrote the fix himself: [#209](https://github.com/Piebald-AI/splitrail/pull/209) "Include Claude Code subagent transcripts" (+428/−100 across seven files), whose description restates the issue's analysis and adopts its central caution — simply lifting the depth cap isn't enough; subagent records have to flow through the same `local_hash` dedup semantics as main transcripts, or you trade an undercount for a double-count. Both fixes shipped together in **splitrail 3.6.1**, which hit Homebrew within a day. On my corpus, the newly visible class is row two of the table above: 16,160 messages and 13.7M output tokens finally entering the totals. A fresh 3.6.1-vs-log reconciliation of the live tree is the obvious next check — part 3 starts there. **Live mutable files are not an audit trail.** If your source can rewrite history, recomputation is not accounting. Append-only ingest with a stable per-record identity (`message.id` , last-write-wins on partials) is cheap insurance — mine is a few hundred lines and a SQLite file. **Reconcile token-exact or you know nothing.** "Close enough" totals hide entire bug classes. It was exact agreement on the scanned subset that turned the remaining gap into two nameable, fixable defects instead of a shrug. **Decompose the gap before you blame anyone.** "Your numbers are lower than mine" is an accusation; *"the gap is exactly deleted-files + an unscanned directory class, here's the table"* is a bug report a maintainer can act on in hours. Both turnarounds here — #200→#204 in eight days, #207→#209 in two — happened because the mechanism arrived with the report. **Frozen snapshot + isolated $HOME is the whole trick for tracker A/B.** Clone the data, pin `HOME` (and `XDG_STATE_HOME` /`XDG_DATA_HOME` — on Linux, state dirs honor XDG and will escape your sandbox; CodeRabbit's review caught that one in my own fixture, fair is fair), and two binaries scanning identical bytes become a controlled experiment.| date (2026) | event | |---|---| | Jul 11 | | Twelve days, two bugs, zero arguments about whose numbers were right — the log settled every question before it became a debate. All of this ran on [TraceGuard](https://github.com/lizhuojunx86/traceguard)'s `routing_audit` module (v1.1.0, "audit evidence layer") — the append-only, message.id-keyed ingest of Claude Code transcripts into a SQLite trace store that part 1 was built on. Stable totals are the substrate; the point of the layer above is **stated-vs-revealed routing analysis, priced per decision** — which model you *said* you'd route to, which one actually ran, and what the difference cost. That's part 3, and my totals finally sit still long enough to write it. (Apparently not a niche itch, either — the splitrail maintainer starred TraceGuard during this exchange and said he'd be interested in the audit part.) `pip install traceguard` — Apache-2.0. The regression fixture is ~600 lines of stdlib Python + bash, merged in [PR #208](https://github.com/Piebald-AI/splitrail/pull/208); the subagent fix shipped in splitrail [3.6.1](https://github.com/Piebald-AI/splitrail/releases).