Writing evidence-linked docs exposed two missing regression tests Shiki Yusuke, the developer behind the open-source CLI agent-cost, used his evidence-docs tool to create evidence-linked documentation for the project. This process exposed two missing regression tests in the codebase, which were added via pull requests without any implementation changes. The gaps were in the cache-write TTL breakdown logic and the pricing-status aggregation logic, where the middle cases were untested. This is a crosspost of the canonical version on GitHub. I run agent-cost https://github.com/shiki-yusuke/agent-cost , a small open-source CLI that reads local Claude Code / Codex CLI usage logs and estimates token cost. I wanted documentation for it more trustworthy than a hand-written README: not prose claiming "the reader handles the TTL cache-write breakdown correctly," but claims that each point at the exact test or source line backing them, checked against real git history so the pointer can't silently go stale. To do that I used a second tool I've been building, evidence-docs https://github.com/shiki-yusuke/evidence-docs , to build a agent-cost : 17 individual observation s grouped into 5 topic s pricing-catalog validation rules, the cache-write lower-bound behavior, unpriced-fact handling, decimal arithmetic for money, and the measure command's v1 contract . Each observation is one statement — a behavior, an invariant, a decision record — with a claim kind , an epistemic status , and one or more provenance entries naming the exact file, test, or spec section it's backed by, plus a content digest of that source at a specific commit.Building that corpus is what surfaced this post's actual finding. To write an honest provenance entry for a behavior claim, I had to point at the test that exercises it — reading the test, not just the source, for every claim. Doing that for the cache-write TTL breakdown logic and the pricing-status aggregation logic turned up two places where the claim I wanted to make "the code does X in this case" was true by reading the code, but had no test asserting it . Both went into docs/claims/gaps.yaml as GAP-01 and GAP-02 , and both became small, scoped pull requests that added regression tests with zero implementation changes. Neither gap was a live bug — that's the whole point of this post. agent-cost 's existing test suite already covered the two extremes of each piece of logic; what was missing was the middle case, the one that's easy to reason yourself past while staring at the code. agent cost/readers/claude.py , parse session facts : when a Claude Code usage event's cache creation field is a dict, the reader computes leftover = cache creation input tokens - ephemeral 5m + ephemeral 1h and emits it as a cache write unknown fact if positive. Existing tests covered a breakdown that exactly accounts for the total, and no breakdown at all — but not a breakdown dict that's agent cost/aggregate.py , STATUS RANK / build rows : a row's pricing status should be the worst status among its facts, ordered unpriced 0 < lower bound 1 < priced 2 . The existing test only mixed an unpriced fact with a priced fact. No test built a row from a lower bound fact e.g. a cache write unknown token, priced as an explicit floor mixed with a plain priced fact to confirm the row lands on lower bound , not priced — the half of the ordering that isn't "obviously" covered by the unpriced case.If either had silently broken in a later refactor — reordering STATUS RANK 's values, or changing the leftover math — nothing in CI would have caught it. The cost-estimation output would have quietly started under- or over-reporting cost floors, without a single red test. The obvious response to "I want good docs for this repo" is: write a good README, or write good docstrings, and be careful. I already had both — agent-cost 's README has a "What this measures, and what it doesn't" section, and price fact has a docstring explaining the lower-bound design decision. Careful prose is necessary but not sufficient, for two reasons that showed up directly here: provenance entry that must name a specific test does.So the fix isn't "write better docs" in the abstract — it's structural: every behavior claim needs a machine-checkable pointer to what backs it, narrow enough that "no test covers this case" becomes visible while writing it, not something discovered later. evidence-docs enforces this as a schema-level invariant on every observation , not a style guideline. epistemic status must be one of a fixed vocabulary from execution verified down to model inference / single source observation / recorded decision — you have to say how strongly the claim was checked, not just assert it. provenance source kind source , test , spec , or review memory , a repo-relative uri , a selector which test/function/section , a content digest , and the repo commit the digest was taken against.Neither field is optional, and both are validated structurally: unknown source kind values are rejected outright, and content digest is checked against the actual git blob at the declared commit, not the working tree. Two of agent-cost 's own observations OBS-004, the cache-write-TTL claim, and OBS-010, the pricing-status-ranking claim exist because writing their provenance forced me to go find "the test that proves this," and in both cases the honest answer was "there isn't one for this specific sub-case" — which is what gaps.yaml records. An evidence-docs corpus is a directory conventionally docs/claims/ with: id-registry.yaml every topic id / observation id used anywhere must be pre-registered here, closing typo/duplicate ID bugs at generate time ; topics/ .yaml and observations/ .yaml one file per topic by convention, not enforced ; and gaps.yaml , a ledger of gaps found gap found: true or explicitly searched for and not found gap found: false , each tagged with a taxonomy — test missing for both GAP-01 and GAP-02 here, independent re search no drift for a third entry, GAP-03, where I re-checked three README pricing claims against the current rates.json and found no drift, recorded as a negative result rather than omitted. evidence-docs validate docs/claims --repo-commit