{"slug": "one-nul-byte-made-14994-signed-records-verify-as-tampered", "title": "One NUL byte made 14,994 signed records verify as tampered", "summary": "A developer building Chron, a local audit log for AI coding sessions, discovered that a single NUL byte in one of 73,526 imported events caused a 14,994-message hash chain to falsely report tampering. The bug arose because SQLite stored 530 bytes but returned only 298 characters on read, so the write-time hash and verification hash operated on different inputs; the developer's fix is to hash only content that can be read back from storage, and to test other characters like lone surrogates for the same failure mode.", "body_md": "I build [Chron](https://www.npmjs.com/package/chron-mcp), a local audit log for AI coding sessions. Every message is hashed into a chain, so if a row is edited after the fact, verification fails and says which row.\n\nLast week I added a path that imports existing Claude Code transcripts. I ran it against my own machine, 33 transcripts, about 70,000 events — and then, instead of spot-checking one session, verified all of them. The biggest one failed:\n\n```\nVerifying 3847fff0 — 14,994 messages\nHash chain\n  ✗ Row 8842: content_hash mismatch — row tampered\n```\n\nNothing had tampered with anything. The rows were minutes old and nothing but my own import had ever written them.\n\nFor a tool whose entire job is telling you whether a record was altered, a false positive is worse than a missed detection. A missed detection is a gap. A false positive teaches people to ignore the alarm.\n\nMy first guess was ordering. Imported transcripts keep the client's line order rather than timestamp order, because 11,591 lines in my corpus carry a timestamp earlier than a line before them. Get that wrong and the chain links up in the wrong sequence.\n\nThat guess was wrong, and the error message said so. A `prev_hash` mismatch means the rows are in the wrong order. A `content_hash` mismatch means this row's stored hash doesn't match this row's stored fields. Ordering was fine. The row itself disagreed with itself.\n\nWhich shouldn't be possible. The code hashes a string and inserts the same string, in the same function, a few lines apart.\n\nSo I looked at what was actually in the row:\n\n```\nSELECT length(content)                  AS chars,\n       length(cast(content AS blob))    AS bytes,\n       instr(content, char(0))          AS nul_at\nFROM messages WHERE id = '2d9746d3…';\nchars  bytes  nul_at\n298    530    299\n```\n\n298 characters. 530 bytes. And a NUL at position 299 — past the end of a string that is only 298 long.\n\nSQLite stored every one of those 530 bytes. It is perfectly happy to hold a NUL inside a TEXT value. What it will not do is pretend the NUL isn't there when something asks for the value as a string. `length()` stops at the NUL. So does the driver, handing the value back to JavaScript.\n\n```\nwrote:  530 bytes\nread:   298 characters, 308 bytes\n```\n\nSo the write hashed 530 bytes of content and the verification hashed the 298 characters that came back. Two different inputs, two different digests, one \"row tampered\".\n\nThe content was tool output. Some program in some session printed a NUL byte, the way programs sometimes do, and it rode all the way into an evidence store. One byte, in one row, out of 73,526 — enough to invalidate a 14,994-event chain, because every row after it inherits the break.\n\nThe tempting fix is to strip the NUL inside the hash function. It fails, and it's worth seeing why:\n\n`sanitize(content)`, store `content` — still 530 bytes, still with the NUL.`sanitize(truncated)`.\n`sanitize` doesn't help, because the two sides aren't hashing the same content in the first place. The stored value is the problem. You have to change what goes into the database, not just what goes into the digest.\n\nThe rule I ended up writing on the wall: **hash what you can read back.** Not what you were handed — what the storage layer will give you again later.\n\nRather than fix the one byte I'd found and move on, I went looking for what else wouldn't survive the trip. This is short enough to run against your own database:\n\n``` js\nimport { createClient } from '@libsql/client';\nimport { createHash } from 'crypto';\n\nconst db = createClient({ url: 'file::memory:' });\nawait db.execute('CREATE TABLE t (id TEXT PRIMARY KEY, body TEXT)');\n\nconst sha = s => createHash('sha256').update(s).digest('hex').slice(0, 12);\nconst cases = {\n  'embedded NUL':        `before${String.fromCharCode(0)}after`,\n  'lone high surrogate': `before${String.fromCharCode(0xd800)}after`,\n  'lone low surrogate':  `before${String.fromCharCode(0xdc00)}after`,\n  'emoji (valid pair)':  'before\\u{1f600}after',\n  'CRLF and tab':        'before\\r\\n\\tafter',\n  'ESC and DEL':         `before${String.fromCharCode(0x1b)}${String.fromCharCode(0x7f)}after`,\n};\n\nfor (const [name, wrote] of Object.entries(cases)) {\n  await db.execute({ sql: 'INSERT INTO t VALUES (?, ?)', args: [name, wrote] });\n  const read = (await db.execute({ sql: 'SELECT body FROM t WHERE id = ?', args: [name] })).rows[0].body;\n  console.log(\n    `${read === wrote ? 'ok    ' : 'BROKEN'} ${name.padEnd(20)} ` +\n    `len ${wrote.length} -> ${read.length}  sha ${sha(wrote)} -> ${sha(read)}`\n  );\n}\n```\n\nOn `@libsql/client` 0.17.3 and Node 23:\n\n``` php\nBROKEN embedded NUL         len 12 ->  6  sha 92e7bd379d66 -> 6db7d803e74f\nBROKEN lone high surrogate  len 12 -> 12  sha a95d623c5792 -> a95d623c5792\nBROKEN lone low surrogate   len 12 -> 12  sha a95d623c5792 -> a95d623c5792\nok     emoji (valid pair)   len 13 -> 13  sha cd9b426a0837 -> cd9b426a0837\nok     CRLF and tab         len 14 -> 14  sha 51daa3413452 -> 51daa3413452\nok     ESC and DEL          len 13 -> 13  sha 9cc6db066164 -> 9cc6db066164\n```\n\nEverything else I threw at it survives. Control characters are fine. CRLF is fine. Emoji are fine, as long as the surrogate pair is intact.\n\nLook at the two surrogate rows again. `read === wrote` is false — the string that came back is not the one that went in. But the hashes match.\n\nI nearly wrote this up as \"NUL and lone surrogates both break the chain.\" They don't, and the reason is more interesting than if they did.\n\nA lone surrogate is not encodable as UTF-8. The driver substitutes U+FFFD on the way through. And Node's UTF-8 encoder performs the *identical* substitution when you hash the string:\n\n``` php\nBuffer.from('a\\uD800b', 'utf8')  ->  61 ef bf bd 62\nBuffer.from('a�b', 'utf8')  ->  61 ef bf bd 62\n```\n\nBoth sides mangle it the same way, so the digests agree and verification passes. The integrity check survives on a coincidence, that two unrelated components happen to implement the same replacement rule.\n\nNothing documents that. Nothing enforces it. Hash bytes instead of a string, move to a runtime whose encoder throws instead of substituting, and the coincidence stops holding. A load-bearing accident is still an accident.\n\nSo I normalise both, for different reasons. The NUL, because it breaks verification today. The surrogate, because storing content the caller never sent is bad on its own terms, and because I would rather not depend on two encoders agreeing by luck.\n\nBoth become U+FFFD rather than being deleted. That's the standard marker for a character that couldn't be represented, and it keeps the fact that something was there.\n\nWriting a test after fixing a bug tells you nothing until you've watched it fail. I reverted the normalisation and ran the new tests against the old behaviour: 11 of 21 failed, including verification through every write path and a twelve-row chain. Put the fix back, all 21 pass.\n\nThat's the part I'd skip if I were in a hurry, and it's the part that tells you the test is real.\n\n**Hash what you can read back.** Any scheme that hashes content and stores it separately has a serialisation boundary in the middle, and the boundary is allowed to change your data. Content-addressed stores, signed records, dedup keys, ETags - same shape, same exposure.\n\n**A silent substitution is worse than an error.** Every layer here behaved reasonably on its own. SQLite stored the bytes. The driver returned a C string. Node's encoder replaced what it couldn't encode. Nothing logged a warning, and the defect surfaced as an accusation against innocent data.\n\n**Verify all of it, once.** This was one row in 73,526. Any sampling strategy misses it. The only reason I found it before a user did is that I got suspicious and checked everything instead of the one session I'd picked.\n\n**And be careful which failures you claim.** I was one draft away from publishing a confident, wrong explanation of the surrogate case. The probe was right there; I just hadn't read its output closely enough.", "url": "https://wpnews.pro/news/one-nul-byte-made-14994-signed-records-verify-as-tampered", "canonical_source": "https://dev.to/sirinivask/one-nul-byte-made-14994-signed-records-verify-as-tampered-3ih0", "published_at": "2026-09-26 19:06:42+00:00", "updated_at": "2026-09-26 19:31:07.411614+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "mlops"], "entities": ["Chron", "Claude Code", "SQLite", "libsql", "npm"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/one-nul-byte-made-14994-signed-records-verify-as-tampered", "markdown": "https://wpnews.pro/news/one-nul-byte-made-14994-signed-records-verify-as-tampered.md", "text": "https://wpnews.pro/news/one-nul-byte-made-14994-signed-records-verify-as-tampered.txt", "jsonld": "https://wpnews.pro/news/one-nul-byte-made-14994-signed-records-verify-as-tampered.jsonld"}}