# You can never backfill a read receipt: a blog comment fixed our AI pipeline

> Source: <https://dev.to/gxcafellc/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline-1ki2>
> Published: 2026-08-31 23:23:55+00:00

Last week we published a post-mortem: our AI reviewer hallucinated a request

that didn't exist, and our producer retried the same document 245 times. A

reader, pm25coder, left a comment that turned out to be the best code review

we've ever received. This post is what we shipped because of it, same day.

Our fix at the time was provenance: record which artifact each output was

made from. The comment pointed at the hole:

"Which artifact it was made from" can still be gamed by the exact failure

it exists to catch — a consumer that records the source id without reading

the content. [...] Make the consumer cite what it actually read. [...]

You can never backfill a read receipt, so it has to be written when the

reading happens.

He was right, and we checked: our approval queue recorded the *path* of

every deliverable. Nothing anywhere proved the enqueuer had ever opened the

file. An empty or deleted file could sit in the human approval queue

looking exactly like a real one.

**1. Read receipts at hand-off.** When the reviewer receives an excerpt, we

now write `{sha256(excerpt), excerptChars, fullChars}`

at the moment of

hand-off. Written at read time, because it cannot be written later.

**2. Fingerprints at enqueue.** A deliverable enters the approval queue only

after the enqueuer actually reads it: content hash and length are stored on

the queue entry. Unreadable or empty → it never enters the queue, and the

reason is archived where a human can see it.

**3. Re-derivable citations.** The queue entry also quotes the reviewer's

verdict line *verbatim*. The nightly audit re-extracts that line from the

review file and compares. A citation that cannot be re-derived from the

artifact means someone wrote it without reading — and it now shows up as a

**mismatch count**, not as an absence. That was the commenter's core point:

an unread input should be a number on a chart, not a blank space.

Worth confessing: our first version skipped any queue entry that lacked a

hash — `if (!entry.sha256) continue;`

— which meant a forged citation

*without* a hash sailed through the very audit built to catch forged

citations. We only found it because we test every check by faking the

failure it should catch, and the fake passed. If you build watchdogs:

break them on purpose before you trust them. Ours needed it, again.

The output-contract checker from this story is free on npm:

[honto-contract](https://www.npmjs.com/package/honto-contract).

Our unattended-operation checklist and three watchdog templates are free

(email-gated):

[Unattended-Operation Kit](https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=harness-kit)

The full production set — now including the payment gate we extracted this

week — is [on the same page](https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=templates-pro).

Honest note: still no customers. Everything above is what we run on

ourselves, measured on our own failures. And thanks again, pm25coder —

comments like yours are why we write these.