An assistant remembers that Sarah works at Acme.
Then Sarah changes jobs.
A month later, you ask:
“Where was Sarah working when we met?”
The assistant confidently answers with her new employer.
It retrieved a real fact. It identified the right person. It may even have selected the most recently updated record.
And it still answered the wrong question.
This is a useful way to think about AI memory failures: sometimes the problem is not forgetting. It is remembering something without preserving the conditions under which it is relevant.
For an engineering design, I would treat validity as a first-class concern—not metadata added after retrieval.
Consider this message:
“I moved to Pune in March.”
Suppose the user says it in June.
There are at least two distinct times to represent:
These timestamps answer different questions.
“Where did I live in April?” asks about the world.
“What did you know about my location in April?” asks about the system’s knowledge at that point.
Treating both as a single message timestamp loses that distinction.
This is the motivation behind a bitemporal representation: separate valid time—when a fact held in the world—from transaction or ingestion time—when the system recorded it. Graphiti’s architecture explicitly distinguishes these timelines and tracks validity and record lifecycle timestamps. arxiv
This is not a claim that every assistant needs a temporal graph database. It is a claim that the data model needs to distinguish questions the product promises to answer.
Imagine these statements arriving over time:
“I work at Acme.”
“I’m joining Nova next month.”
“My start date moved back two weeks.”
“Actually, I decided to stay at Acme.”
A memory pipeline that simply overwrites current_employer risks collapsing several different concepts:
My proposed design rule is simple:
Do not turn every newly extracted sentence into an immediately valid replacement fact.
A plan is not an event. A correction is not always a new state. A later message can change how an earlier message should be interpreted.
Temporal memory systems such as Graphiti preserve superseded facts through invalidation rather than simply deleting them, allowing current and historical relationships to remain distinguishable. getzep
But preserving history is only part of the design. The reader must also choose the right historical state for the question.
I would separate three layers in a memory implementation.
What was actually said?
Keep the source message, speaker, timestamp, and enough surrounding context to interpret it.
What does the system think the source claims?
For example:
{
"subject_id": "person_sarah_01",
"predicate": "employed_by",
"object_id": "company_acme_01",
"assertion_type": "reported_state",
"recorded_at": "2026-06-10",
"valid_from": null,
"valid_to": null,
"source_message_id": "message_184"
}
This is an illustrative schema, not a complete temporal database design.
The null dates matter. If the source does not establish when employment started, the extractor should not invent a start date.
What should the assistant currently believe?
A current-state view can select the assertions relevant to “Where does Sarah work now?”
A historical view can select evidence relevant to “Where did Sarah work when we met?”
These views should point back to the underlying evidence.
The architectural goal is to make interpretations revisable without losing the information used to produce them.
Now introduce two people named Sarah.
One is your colleague. The other is your sister.
A name alone cannot distinguish them in this example. But “name plus relationship” is not necessarily a permanent identifier either: colleagues become friends, managers change, and relationships can overlap.
I would use stable internal entity IDs, with names and relationships stored as evidence-backed attributes.
For ambiguous references, the write path should have more than two options.
Instead of only:
Create entity
or
Merge into existing entity
allow:
Create entity
Link with sufficient evidence
Hold as unresolved
Ask for clarification
A system that can preserve uncertainty does not have to turn every ambiguous mention into a confident merge.
Consider four questions:
They concern the same person and topic, but they ask for different evidence.
For this design, I would represent query intent explicitly enough to distinguish:
The retrieval pipeline could then look like this:
Question
↓
Entity resolution
↓
Temporal and question-intent interpretation
↓
Candidate retrieval
↓
Validity and provenance checks
↓
Evidence selection
↓
Answer or abstention
This is a proposed decomposition, not an argument that every box requires a separate model call.
The important part is that “most similar” and “appropriate evidence” are not treated as interchangeable.
Suppose a fact concerns both Sarah and Acme.
It should be discoverable through either entity.
For a structured implementation, I would prefer one canonical fact record referenced by both entity indexes rather than two independently editable copies:
Sarah index ──┐
├── fact_271
Acme index ───┘
That gives corrections one authoritative target.
Materialized copies may still be useful for performance, but then their consistency needs to be an explicit engineering responsibility.
A chapter metaphor can make navigation intuitive. It should not accidentally dictate an update model that is difficult to maintain.
Imagine a memory system with three stages:
Conversation → Extraction → Filing → Retrieval
A wrong answer could originate anywhere:
Improving the retriever will not reveal which of these happened.
Before choosing the next model to train, I would run an ablation:
| Condition | Extraction | Filing |
|---|---|---|
| A | Human-verified | Human-verified |
| B | Model-generated | Human-verified |
| C | Human-verified | Model-generated |
| D | Model-generated | Model-generated |
Keep the retrieval procedure fixed.
Condition A is a reference condition for retrieval with verified inputs—not a theoretical upper bound.
The other conditions help investigate which upstream errors are responsible for the observed failures.
The evaluation should report those failures separately, rather than reducing every error to “memory recall.”
A useful test set should contain more than questions with known, neatly stored answers.
I would include:
For each case, define acceptable behavior in advance.
Sometimes the correct output is a fact.
Sometimes it is a qualified answer.
Sometimes it is:
“I know Sarah changed employers, but I don’t have enough information to determine where she worked on that date.”
An evaluation that always requires a factual answer cannot tell you whether the system handles missing evidence appropriately.
Retrieval evaluation is useful, but it does not automatically establish the performance of the complete assistant.
For a proposed system, I would track at least three groups of measurements.
A selective reader may reduce context size, but that remains an empirical question until the full reading loop is integrated and measured.
Likewise, an extra retrieval step might save prompt tokens while adding latency. The relevant trade-off belongs in the end-to-end results, not only in an architectural diagram.
Semantic retrieval remains useful. It can help locate evidence when the user’s wording differs from the stored text.
But my design recommendation is to place it inside a broader lifecycle:
Capture
→ Interpret
→ Resolve identity
→ Record provenance
→ Establish validity
→ Retrieve
→ Verify
→ Correct
Existing systems already address parts of this lifecycle. Graphiti documents temporal invalidation and historical relationships; Mem0 exposes a memory-history interface for tracking how an individual memory has changed. Neither should be reduced to “just a pile of embeddings.” help.getzep
The useful engineering question is therefore not simply:
“How do we give an assistant memory?”
It is:
“What evidence does this assistant preserve, how does it revise that evidence, and under what conditions is it allowed to use it?”
An assistant that remembers yesterday’s fact is useful.
An assistant that can explain whether the fact is still true—and recognize when it cannot tell—is a more demanding system to build.