cd /news/ai-agents/my-agent-s-memory-contradicted-itsel… · home › topics › ai-agents › article
[ARTICLE · art-168963] src=dev.to ↗ pub= topic=ai-agents verified=true sentiment=↑ positive

My agent's memory contradicted itself and nobody noticed. I built a read-only audit CLI.

A developer built memrot, an MIT-licensed read-only CLI that audits agent memory stored in Relate, Neo4j's embedded graph database for AI agents, rather than trusting it. The tool runs seven deterministic checks inside the store — including contradictions between current facts, superseded facts still served, stale facts, missing provenance and near-duplicates — and returns exit codes (0 clean, 1 error findings, 2 usage/store error) usable directly in CI. On a seeded demo memory with seven planted faults and three clean controls, the audit reported exactly 9 findings, 5 errors and 4 warnings.

by read7 min views3 publishedNov 13, 2026

Last week I asked a simple question about an agent's memory: at one point in February, what did it believe the database engine was? The answer came back with two values at once. Postgres 15 and Postgres 16. Both marked current. Both believed.

The agent had not reasoned badly. Its memory was rotten, and nothing in the loop ever checked the store itself. A limit changes and both values stay current. An upgrade supersedes a fact that is never marked superseded. A TTL expires and nobody notices. A fact arrives with no record of where it came from. The agent then acts on stale or contradictory memory with full confidence, and the failure looks like a reasoning bug when it is really a data bug.

I built a small tool that audits the memory instead of trusting it. It is called memrot, and it is a read-only CLI over a Relate store (Neo4j's embedded graph database for AI agents). MIT licensed. No server, no LLM, no network, no writes to the store it audits.

I re-verified the rivals live on 13 November 2026 before writing a word of this:

Every rival stores memory. None of them lets an operator open a store read-only, ask "what in here is wrong", and get a machine-checkable answer with an exit code. That is the wedge, and memrot is the audit layer they do not have.

memrot runs seven deterministic checks, each one a Cypher query or a vector search executed inside Relate, not application-side logic:

# Check Rule Severity
1 Contradiction Two current facts, same subject + predicate, different object, overlapping validity windows error
2 Superseded-but-served A fact superseded by another current fact, still markedcurrent error
3 Stale A current fact past itsvalidTo , or older than--stale-days (default 90) with no corroboration and no newer fact on the same subject + predicate warning
4 Orphaned entity An Entity with no relationships at all warning
5 Provenance-less fact A Fact with noFROM_EPISODE edge: no record of where it came from error
6 Validity-order violation validFrom aftervalidTo , or a superseding fact created before the fact it supersedes error
7 Near-duplicate (lexical) Two facts whose trigram embeddings score at or above 0.91: the same claim recorded twice under different wording warning

The answer comes back as an exit code a shell or CI step can use directly: 0 = clean, 1 = error-severity findings, 2 = usage error or store error. Warnings alone do not fail the audit.

memrot seed demo.rl
memrot audit demo.rl
memrot audit demo.rl --format json
memrot audit demo.rl --as-of 2026-03-01 --stale-days 30
memrot believed-at demo.rl --at 2026-02-15
memrot believed-at demo.rl --at 2026-02-15 --subject database
memrot why demo.rl f-db-15

believed-at is time travel: what did the agent believe at this moment. why shows a fact's provenance, its supporters, its contradictions, and the fact that supersedes it.

The GIF above is a real terminal session, not a mockup. memrot seed writes a small fictional migration memory: 11 facts, 6 entities, 8 episodes, 7 planted faults and 3 clean controls. The audit finds exactly 9 findings, 5 errors and 4 warnings:

$ memrot audit examples/demo.rl --as-of 2026-11-13

memrot audit: examples/demo.rl (as of 2026-11-13)

- Contradictory facts: 2 finding(s) [error]
  ERROR  database.engine has two current values in overlapping windows: "Postgres 15" (f-db-15) and "Postgres 16" (f-db-16).
  ERROR  queue.max_payload has two current values in overlapping windows: "4000 bytes" (f-queue-4000) and "8000 bytes" (f-queue-8000).
- Superseded facts still served: 1 finding(s) [error]
  ERROR  database.engine = "Postgres 15" (f-db-15) was superseded by f-db-16 but is still marked current, so it can still be served.
- Stale facts: 2 finding(s) [warning]
  WARN  cache.ttl = "300s" (f-cache-ttl) expired on 2026-06-01 and is still current. It is also old (recorded 2026-01-05T10:00:00Z) with no corroboration and no newer fact on the same subject and predicate.
  WARN  feature.flag = "on" (f-feature-flag) expired on 2026-04-01 and is still current. It is also old (recorded 2026-05-02T09:30:00Z) with no corroboration and no newer fact on the same subject and predicate.
- Orphaned entities: 1 finding(s) [warning]
  WARN  Entity "LegacyCron" (e-legacycron, kind service) has no relationships: nothing references it and it references nothing.
- Provenance-less facts: 1 finding(s) [error]
  ERROR  deploy.region = "eu-west-1" (f-deploy-region) has no episode: there is no record of where this fact came from.
- Validity-order violations: 1 finding(s) [error]
  ERROR  feature.flag = "on" (f-feature-flag) has validFrom 2026-05-01 after validTo 2026-04-01: the validity window is empty.
- Near-duplicate facts (lexical): 1 finding(s) [warning]
  WARN  database.engine = "Postgres 16" (f-db-16) and database.db_engine = "PostgreSQL 16" (f-db-16b) are near-duplicates (lexical cosine 0.93): the same claim recorded twice under different wording.

9 finding(s): 5 error(s), 4 warning(s)
audit: FAILED (exit 1)

Time travel on the same store shows the contradiction was there from the start:

$ memrot believed-at examples/demo.rl --at 2026-02-15

  cache.ttl = "300s" (f-cache-ttl)
  database.db_engine = "PostgreSQL 16" (f-db-16b)
  database.engine = "Postgres 15" (f-db-15)
  database.engine = "Postgres 16" (f-db-16)
  queue.max_payload = "8000 bytes" (f-queue-8000)

5 fact(s) believed at 2026-02-15.

In February the agent already believed both Postgres 15 and Postgres 16. The March payload change is not believed yet. And memrot why demo.rl f-db-15 traces the bad fact back to its episode ("DB still on Postgres 15", session 2026-02-01), names its superseder (f-db-16) and its contradiction (f-db-16: "Postgres 16").

Every check is asserted by exact fact id in the test suite (15/15 tests pass, against seeded stores with known ground truth), so the audit is proved by CI on every push, not just by the demo.

The npm package is not published yet. The name memrot is free on npm, but publishing is blocked on the owner's npm token, so for now install from source:

git clone https://github.com/techdevfun/memrot.git
cd memrot
npm install
npm run build

Requires Node.js 20 or newer. The only runtime dependency is @relate/relate 1.0.1, pinned exact. A clean install from source was measured on 13 November 2026 at about 8 seconds (npm install + npm run build, no cache). Run it from the repo root, or npm link to make memrot available everywhere.

One thing to know before you audit a live agent: memrot opens the store read-only and never writes to it, but Relate has a lock rule I verified live. A .rl file held by a running writer cannot be opened read-only, and relate clone cannot copy it either. Both fail, and memrot prints the remedy instead of the raw error. A directory store can be opened read-only beside a live writer. So: for a .rl file, stop the agent first, then audit; or snapshot it after the writer exits and audit the snapshot.

Also worth knowing: the relate CLI ships with telemetry enabled by default. memrot itself sends nothing (its CI installs only the @relate/relate library, never the CLI), but if you install the relate CLI, run relate telemetry off first.

Honesty first: memrot is the audit layer, not a replacement memory.

If you need semantic judgement or retrieval, pick a rival. The README says so, and means it. If you need a standalone answer to "what in this store is wrong, and can CI fail on it?", try memrot.

memrot index command to build embeddings on a copy of a store. A single .rl file is the whole point: a memory an agent carries as one file, auditable anywhere, no server to deploy. Directory mode exists for live auditing beside a writer; :memory: mode exists for tests. The audit runs fully offline: an audit that needs a model call cannot run in CI on every commit.

If your agent has persistent memory, run one audit before the next bad decision:

Feedback welcome in the repo issues. The roadmap (embedder-backed semantic near-duplicates, memrot index, --fail-on warning) is in the README.

── more in #ai-agents 4 stories · sorted by recency
── more on @memrot 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/my-agent-s-memory-co…] indexed:0 read:7min 2026-11-13 · —