# Five things to check your agent memory store against

> Source: <https://dev.to/sovantica/five-things-to-check-your-agent-memory-store-against-427k>
> Published: 2026-09-01 10:02:38+00:00

An agent-memory bug can look nothing like one. It looks like confidence: the agent remembers an old address, keeps a preference the user changed months ago, or retrieves a fact that should have expired weeks back. Nothing crashes. The answer is just wrong - and the reason is buried in a store that kept growing because every write looked harmless at the time.

This article is about that layer: what goes wrong *after* agent memory starts working. Five failure modes show up often enough to be worth checking any store against, so each one below carries the question to put to yours. They're not equally bad - three of them are correctness problems, the fourth is a security one - and I'll be honest about which parts a memory layer can actually fix versus merely make visible.

One bias up front, so you can discount for it: Engrava (the library I work on) has no LLM in its write path. That's relevant because when nothing generative runs on write, what the store does is decided by code you can read - so I can be specific about where it breaks, and, where it can, design against that in the data model rather than hoping a bigger model smooths it over. Where it *can't*, I'll say so.

You ask the store for "high-confidence facts about the user" and get back entries tagged with confidence scores or categories that were never true - invented somewhere on the way in.

This one is specific to a design choice: running an LLM *on write*, to extract entities or tag records. It's convenient, and it adds a second place for the model to hallucinate - except now the hallucination is persisted as structured metadata you'll trust and query against later. The bug isn't in retrieval; it's baked into the row. So the check is narrower than whether an LLM touches the write path at all: it's whether model output gets persisted as fields you later query against - and if it does, what validates it before it lands, and whether anything marks it as model-written.

Engrava sidesteps this by not having a model in the write path at all. A `Thought`

is a frozen, validated record - you don't mutate it in place, you call `evolve()`

and get a new validated instance. Whatever metadata exists was written by your code, deterministically, or it isn't there. You can still *use* an LLM to decide what's worth storing; the storing itself is plain code you can read. It's a smaller capability than "the memory understands your data," and that's the point: the store is not inventing any of it. What your code hands it can still be wrong, and Engrava will store that faithfully.

A user moves from Berlin to Lisbon. They tell the agent. Six weeks later the agent tells a colleague they're in Berlin - cheerfully, with no hint anything is wrong.

The first time you see this in a real agent, it doesn't look like a database bug. It looks like the model being strangely sure about something that used to be true. What actually happened is duller: both facts are in the store, nothing recorded that the second *replaced* the first, and the retriever picked the older one because it scored higher. In an insert-only store a new fact doesn't retract an older one; they just coexist, and "which one is true now" is information nothing in the store wrote down.

Mem0's [issue #4896](https://github.com/mem0ai/mem0/issues/4896) names this directly - *"ADD-only architecture doesn't implement conflict resolution for semantically similar memories"* - and was closed as `not planned`

. A [related issue #4536](https://github.com/mem0ai/mem0/issues/4536) asks for contradiction handling on the add path. (This doesn't mean Mem0 is broken - it means contradiction becomes a real product surface once agents run long enough, and it's genuinely hard.) So the thing to ask about your own store is what happens on the second write: does it keep both and let ranking sort it out, or does something record that one replaced the other?

Engrava's answer is to make "true *when*" part of the model. 0.4.0 added **bi-temporal valid-time**: each fact carries not just when it was written but when it held in the world. You don't delete Berlin - you set its `valid_until`

to the move, and Lisbon's `valid_from`

from it. Ask "as of now" and you get Lisbon; ask for the history and the supersession is right there, auditable. Keep the record, move the truth.

Two limits on that, because it is easy to read more into it than it says. Engrava gives you the representation and the history; it does not notice the contradiction for you. If nothing tells the store that Lisbon supersedes Berlin, both facts sit there and "as of now" answers as of a timestamp, not as of the truth. And detection is not a problem nobody has taken on: cognee, for one, ships a [contradiction check](https://github.com/topoteretes/cognee/blob/v1.5.3/cognee/tasks/graph/detect_contradictions.py) as an optional step in its ingest pipeline. That is a real capability and we don't have it. Whether you want it is the trade-off from section 1 again, because the check is a model pass on the write path.

Six months in, the store has piled up observations that mattered for exactly one turn and never again. The signal you want is still in there, outnumbered by them.

The same ask keeps resurfacing across frameworks ([smolagents #901](https://github.com/huggingface/smolagents/issues/901), [Agno #2500](https://github.com/agno-agi/agno/issues/2500), [Letta #957](https://github.com/letta-ai/letta/issues/957)), and the honest summary is: memory grows unbounded unless something prunes it, and real consolidation is hard. It's worth finding out what in your store ever takes a record back out of the retrievable set - an expiry, a state you can set, a pass that runs on its own - and whether that is the store doing it or you remembering to. I won't claim Engrava has *solved* forgetting - it hasn't.

What it does give you is a lifecycle you can act on. Every thought has an explicit, enforced state - `CREATED -> ACTIVE -> DONE -> ARCHIVED`

, with invalid transitions rejected at the type level - plus an optional TTL (`expires_at`

) for things that should age out on their own. That's not "better retrieval"; it's *control* - a structural way to say "this is working memory," "this is archived," "this expires Friday," so the store reflects what's live instead of everything it ever saw. (There's also a deterministic background consolidation pass; what it's actually worth is a benchmark question, and I'm not going to wave a number at you I haven't published - that's its own article.)

The first three are correctness problems, and what they cost you depends on what your agent is for. This one is different in kind: it turns into a security problem the moment two things you don't equally trust write into the same store.

Run a fleet of agents that don't equally trust each other over one unscoped store, and a single bad actor - buggy, adversarial, or just wrong - writes data the rest of them can retrieve. One weak link, and the bad record is sitting in the namespace they all read from. CrewAI's [#2584](https://github.com/crewAIInc/crewAI/issues/2584) asked for memory distinguished by a custom key and was closed `not planned`

; in that design the caller stays responsible for scoping. That isn't CrewAI being careless - a framework that hands you the store and lets you scope it yourself is a defensible position, and it becomes a problem only when the agents sharing that store stop trusting each other. Where there is no first-class boundary between them, that is where memory poisoning lives. Some hosted memory products have added per-resource authorization since; if you are on one, find out what it does by default. The question isn't whether your store can be scoped, it's what separates two agents that don't trust each other before anyone has scoped anything.

Engrava puts that boundary in the data model. `EngravaManager`

hands each service its own `.db`

file - separate store, separate journal, nothing mutable shared between them. One agent's writes don't land in another agent's store unless you wire them together deliberately. Be precise about what that buys you: it is a boundary in the data model, not an access-control system. Two processes that can both read the directory can both open both files, and file permissions are still yours to set. It also won't stop an agent from poisoning *its own* memory. What it does is keep one agent's bad writes out of the others' stores by default.

An agent surfaces an irrelevant memory at the worst possible moment and you're left with two questions you can't answer: what's actually in the store, and why did the retriever rank *this* over that?

If a memory layer gives you `add()`

and `search()`

and not much in between, the tracing and the observability have to be built *around* the store rather than into it. That is not the only option on offer: hash-chained provenance logs with a verifier you can call ship in paid audit tiers and in open-source memory layers too - cognee's is [Apache-2.0 and one pip install away](https://github.com/topoteretes/cognee/blob/v1.5.3/cognee/modules/provenance/manager.py). Two questions for your own store, then: what does it record about a write that you can read back later, and what does a result tell you about why it ranked where it did. Worth knowing the answers before you build that layer outside it.

Engrava keeps two things in the box. The **CognitiveJournal** is a SHA-256 hash-linked log of every mutation - what changed, when, which service wrote it, replayable, and tamper-evident in the sense a hash chain gives you: an edit somewhere in the middle breaks the chain, provided you have a record of the head to check the chain against. It is not a signature. The service name in an entry is whatever the writing process declared it to be, and the chain does not prove who produced it. And hybrid search hands back part of its own working: vector, keyword (BM25) and recency are fused into one score, and the result carries that score per hit plus the set of backends that were available for the query. Be precise about what that is. It tells you which backends were available for the query and how strongly a hit scored overall. It does not tell you which of them returned that hit or what each contributed to its score, and it is not a claim that the ranking is better - better is a benchmark question, and I'm keeping those for when there are numbers to stand behind.

The useful property isn't that Engrava prevents every bad memory. It doesn't. The useful property is that bad memory stays *inspectable*: when it was written, what changed, which service name the writing process put on it, and whether the journal still verifies. When something does go wrong, you can open the store and see what happened instead of guessing at a model's mood.

Engrava is a deterministic, auditable, local-first store, and it does not fix all five of these. The first is kept out of Engrava's own write path, though Engrava will still store bad metadata your code hands it. The second and third get a data model you can act on, not detection. The fourth is contained, not solved. The fifth is half done: the journal is complete, the ranking explanation isn't. Whatever store you end up on, that is the list worth checking it against.

`pip install engrava`

The code and the architecture behind it: [github.com/sovantica/engrava](https://github.com/sovantica/engrava)
