{"slug": "you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline", "title": "You can never backfill a read receipt: a blog comment fixed our AI pipeline", "summary": "A developer's AI pipeline was fixed after a blog commenter, pm25coder, pointed out that provenance records could be gamed by consumers that record source IDs without reading content. The team implemented read receipts at hand-off, content fingerprints at enqueue, and re-derivable citations, ensuring unread inputs show up as mismatch counts. The developer also confessed that their first audit version skipped entries lacking a hash, allowing forged citations to pass, and emphasized testing watchdogs by faking failures.", "body_md": "Last week we published a post-mortem: our AI reviewer hallucinated a request\n\nthat didn't exist, and our producer retried the same document 245 times. A\n\nreader, pm25coder, left a comment that turned out to be the best code review\n\nwe've ever received. This post is what we shipped because of it, same day.\n\nOur fix at the time was provenance: record which artifact each output was\n\nmade from. The comment pointed at the hole:\n\n\"Which artifact it was made from\" can still be gamed by the exact failure\n\nit exists to catch — a consumer that records the source id without reading\n\nthe content. [...] Make the consumer cite what it actually read. [...]\n\nYou can never backfill a read receipt, so it has to be written when the\n\nreading happens.\n\nHe was right, and we checked: our approval queue recorded the *path* of\n\nevery deliverable. Nothing anywhere proved the enqueuer had ever opened the\n\nfile. An empty or deleted file could sit in the human approval queue\n\nlooking exactly like a real one.\n\n**1. Read receipts at hand-off.** When the reviewer receives an excerpt, we\n\nnow write `{sha256(excerpt), excerptChars, fullChars}`\n\nat the moment of\n\nhand-off. Written at read time, because it cannot be written later.\n\n**2. Fingerprints at enqueue.** A deliverable enters the approval queue only\n\nafter the enqueuer actually reads it: content hash and length are stored on\n\nthe queue entry. Unreadable or empty → it never enters the queue, and the\n\nreason is archived where a human can see it.\n\n**3. Re-derivable citations.** The queue entry also quotes the reviewer's\n\nverdict line *verbatim*. The nightly audit re-extracts that line from the\n\nreview file and compares. A citation that cannot be re-derived from the\n\nartifact means someone wrote it without reading — and it now shows up as a\n\n**mismatch count**, not as an absence. That was the commenter's core point:\n\nan unread input should be a number on a chart, not a blank space.\n\nWorth confessing: our first version skipped any queue entry that lacked a\n\nhash — `if (!entry.sha256) continue;`\n\n— which meant a forged citation\n\n*without* a hash sailed through the very audit built to catch forged\n\ncitations. We only found it because we test every check by faking the\n\nfailure it should catch, and the fake passed. If you build watchdogs:\n\nbreak them on purpose before you trust them. Ours needed it, again.\n\nThe output-contract checker from this story is free on npm:\n\n[honto-contract](https://www.npmjs.com/package/honto-contract).\n\nOur unattended-operation checklist and three watchdog templates are free\n\n(email-gated):\n\n[Unattended-Operation Kit](https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=harness-kit)\n\nThe full production set — now including the payment gate we extracted this\n\nweek — is [on the same page](https://gxcafe.co.jp/harness-kit/?utm_source=devto&utm_medium=article&utm_campaign=templates-pro).\n\nHonest note: still no customers. Everything above is what we run on\n\nourselves, measured on our own failures. And thanks again, pm25coder —\n\ncomments like yours are why we write these.", "url": "https://wpnews.pro/news/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline", "canonical_source": "https://dev.to/gxcafellc/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline-1ki2", "published_at": "2026-08-31 23:23:55+00:00", "updated_at": "2026-08-31 23:53:03.634654+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "developer-tools", "ai-safety"], "entities": ["pm25coder", "honto-contract", "GX Cafe"], "alternates": {"html": "https://wpnews.pro/news/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline", "markdown": "https://wpnews.pro/news/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline.md", "text": "https://wpnews.pro/news/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline.txt", "jsonld": "https://wpnews.pro/news/you-can-never-backfill-a-read-receipt-a-blog-comment-fixed-our-ai-pipeline.jsonld"}}