{"slug": "memory-is-not-written-it-s-audited", "title": "Memory Is Not Written. It's Audited.", "summary": "Wallaby Token has released reconcile.py, a zero-dependency audit script in its open-source wallaby-agent-rules repository that diffs an agent's dated project log against its current-state file to flag contradictions and unverifiable \"rumor\" closures. The script uses a single regex of closure words rather than semantic judgment, reports candidates for human review, and follows the same exit-code contract as the project's health check (0 clean, 1 findings). New installs now build six memory files instead of five, while existing users copy the script in as scripts/reconcile.py and run it weekly.", "body_md": "Here is a failure mode nobody screenshots. Your project log says the checkout redesign shipped on Tuesday. Your current-state file still lists it as in flight, waiting on assets. Both files are \"your AI's memory.\" One of them is lying, and nothing in the system is going to tell you which.\n\nThe files are plain text: a dated log of what happened, a current-state file, and a few rule files an agent reads at startup. No database, no service. That is the point, and also the problem.\n\nWe found our own version of this the unglamorous way: not by noticing, but by writing a script that notices. It reads the dated log, reads the state file, and diffs what the one claims against what the other shows. The first run flagged entries we would have sworn were accurate. That is the entire argument of this post: **write discipline always loosens, so memory needs an audit layer, not more discipline.**\n\nDisclosure: Wallaby Token operates an OpenAI-compatible inference API. The memory system described here is open source, MIT licensed, and is the exact setup our own agents run on.\n\nOnce we started looking, the drift came in exactly two shapes:\n\n**The contradiction.** The log records a closure (shipped, fixed, merged, published) and the state file still carries the same topic as open work. Usually nobody updated the state file, because updating it was part of the closeout, and the closeout got skipped on a Friday. The log is append-only and honest; the state file is edited and forgetful. Left alone, the gap compounds until the state file is a museum of things you finished months ago.\n\n**The rumor.** The log claims done, but points at nothing: no file, no ticket, no link. A closure you cannot verify is indistinguishable from one that never happened. This one is subtler, because the entry feels complete when you write it. Six weeks later, \"fixed the retry bug\" with no pointer is a rumor you told yourself.\n\nBoth are boring. That is what makes them dangerous: there is no dramatic failure, just a slow divergence between what your records claim and what is true.\n\nThe obvious fix is a sterner resolution to keep the files in sync. We tried that. It works for about two weeks. We know, because our audit script is how we found out it had stopped working.\n\nSo the rule we now run is: the human ritual writes, the machine audits. The closeout ritual (from our [last post](https://wallabytoken.com/blog/p/closeout-ritual?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited)) is still the entry point; it sorts each session's output into memory, log, state, or the bin. But rituals have bad weeks, so a scheduled script checks the results. And it checks the one thing the ritual cannot check about itself: whether the story the log tells and the story the state file tells are still the same story.\n\nThe script is deliberately dumb. It does keyword matching, not semantic judgment, and it reports candidates rather than verdicts; a human decides which file is stale.\n\n```\nCLOSURE_WORDS = re.compile(\n    r\"\\b(shipped|ships|done|closed|closes|fixed|fixes|merged|published|\"\n    r\"released|resolved|launched|completed|landed|deployed|closed out)\\b\", re.I\n)\n```\n\nThat regex is the whole semantic layer. No model, no embeddings, no configuration file. Dumb is the feature: a smarter audit would need tuning, and tuning would grow its own maintenance ritual. A dumb one just runs.\n\nThe audit ships today in [wallaby-agent-rules](https://github.com/Dawncoral/wallaby-agent-rules) as `templates/reconcile.py`: zero dependencies, same exit-code contract as the health check (0 clean, 1 findings), two scans:\n\nNew installs get it by default: the L0 and L1 prompts in `PROMPT.md` now build six files instead of five. Existing installs need exactly one step: copy `templates/reconcile.py` into your project as `scripts/reconcile.py` and run it weekly next to the health check. The L2 upgrade check detects v3.1 by the file's presence, and like `health_check.py`, the script is scaffolding: it holds none of your content, so upgrades can offer to swap it outright.\n\nA keyword diff will flag some false positives: a closure whose topic overlaps unrelated open work, a pointer written in a format the script doesn't recognize. We accept that trade knowingly. The audit's job is to make drift *visible*, not to be right about it. A candidate you dismiss in three seconds costs three seconds. A contradiction nobody surfaces costs you the week you spent trusting a file that had quietly become fiction.\n\nThe harder limit is the one from the last post, unchanged: none of this survives if you never run it. Hang the audit on the same schedule as the health check, and let the machine be the part of the system that never gets tired.\n\nAn audit makes drift visible. It does not stop it, and after enough weekly findings you stop treating them as incidents and start reading them as a design signal. Two files can disagree because nothing forces them to agree at the moment of writing.\n\nSo we moved every memory change behind one entrance. Twelve write surfaces go through a single writer with a diff preview and a test suite of 37 cases. The source of truth is an append-only event stream; the SQLite index is derived, and we rebuilt 1,263 of 1,263 records from it in one command to prove it. Corrections are appended too: when 403 timestamps turned out to be missing seconds, we added correction events instead of editing history.\n\nThe validators run in shadow first. They report for two weeks before they are allowed to reject a write. A check that can block on day one is a check nobody trusts.\n\nThe reconcile audit is the third layer of the open stack: the file conventions and health check from v2, the closeout ritual from [v3](https://wallabytoken.com/blog/p/closeout-ritual?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited), and now the consistency audit, all in [wallaby-agent-rules](https://github.com/Dawncoral/wallaby-agent-rules), MIT licensed, all running daily on our own project. The design principles are in [AI remembers, you find](https://wallabytoken.com/blog/p/ai-remembers-you-find?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited); the maintenance layer is in [Your AGENTS.md is rotting](https://wallabytoken.com/blog/p/agents-md-maintenance?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited). And in 1.0.0 the stack grew a write-side counterpart to this audit: [a probation period for new memories](https://wallabytoken.com/blog/p/trust-building-mode?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited), where the agent proposes and you approve.\n\nOur internal version of this script checks more (size budgets, decision ledgers, registration drift), because our workspace grew those needs. What we publish is the part every project needs on day one: the diff between what you said you did and what your files say is true. We also outsourced one audit on purpose: [we paid Claude to review our own blog](https://wallabytoken.com/blog/p/claude-audited-our-blog?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited) — a second pair of eyes with no stake in our conclusions.\n\nNew here? The fastest path is the L0 prompt in [PROMPT.md](https://github.com/Dawncoral/wallaby-agent-rules/blob/main/PROMPT.md): paste two lines, and the audit ships with the install. Already running the system? Copy `templates/reconcile.py` into `scripts/` and run it once today. If it comes back clean, congratulations; your log and your state file agree. If it doesn't, you just found out what the audit is for.\n\nRelated: [Your rule file says \"later.\" That's why it never happens.](https://wallabytoken.com/blog/p/rule-file-says-later?utm_source=devto&utm_medium=post&utm_campaign=memory-is-audited) — three incidents where one deferral word was the whole bug.", "url": "https://wpnews.pro/news/memory-is-not-written-it-s-audited", "canonical_source": "https://dev.to/dawncoral/memory-is-not-written-its-audited-4oh8", "published_at": "2026-10-22 16:09:57+00:00", "updated_at": "2026-10-22 16:13:27.865436+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools"], "entities": ["Wallaby Token", "wallaby-agent-rules", "Dawncoral", "reconcile.py", "health_check.py"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/memory-is-not-written-it-s-audited", "markdown": "https://wpnews.pro/news/memory-is-not-written-it-s-audited.md", "text": "https://wpnews.pro/news/memory-is-not-written-it-s-audited.txt", "jsonld": "https://wpnews.pro/news/memory-is-not-written-it-s-audited.jsonld"}}