{"slug": "your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong", "title": "Your AI Assistant Remembered the Fact. It Still Got the Answer Wrong", "summary": "A developer proposes treating validity as a first-class concern in AI assistant memory design, arguing that many memory failures stem not from forgetting but from remembering facts without preserving the conditions under which they were true. The design separates valid time (when a fact held in the world) from ingestion time (when the system recorded it), keeps source messages, extracted assertions, and current-belief views as distinct layers, and uses stable internal entity IDs rather than names or relationships as identifiers. The approach draws on Graphiti's bitemporal architecture, which tracks validity and record lifecycle timestamps and invalidates superseded facts rather than deleting them.", "body_md": "An assistant remembers that Sarah works at Acme.\n\nThen Sarah changes jobs.\n\nA month later, you ask:\n\n“Where was Sarah working when we met?”\n\nThe assistant confidently answers with her new employer.\n\nIt retrieved a real fact. It identified the right person. It may even have selected the most recently updated record.\n\nAnd it still answered the wrong question.\n\nThis is a useful way to think about AI memory failures: sometimes the problem is not forgetting. It is remembering something without preserving the conditions under which it is relevant.\n\nFor an engineering design, I would treat **validity** as a first-class concern—not metadata added after retrieval.\n\nConsider this message:\n\n“I moved to Pune in March.”\n\nSuppose the user says it in June.\n\nThere are at least two distinct times to represent:\n\nThese timestamps answer different questions.\n\n“Where did I live in April?” asks about the world.\n\n“What did you know about my location in April?” asks about the system’s knowledge at that point.\n\nTreating both as a single message timestamp loses that distinction.\n\nThis is the motivation behind a bitemporal representation: separate valid time—when a fact held in the world—from transaction or ingestion time—when the system recorded it. Graphiti’s architecture explicitly distinguishes these timelines and tracks validity and record lifecycle timestamps. [arxiv](https://arxiv.org/html/2501.13956v1)\n\nThis is not a claim that every assistant needs a temporal graph database. It is a claim that the data model needs to distinguish questions the product promises to answer.\n\nImagine these statements arriving over time:\n\n“I work at Acme.”\n\n“I’m joining Nova next month.”\n\n“My start date moved back two weeks.”\n\n“Actually, I decided to stay at Acme.”\n\nA memory pipeline that simply overwrites `current_employer` risks collapsing several different concepts:\n\nMy proposed design rule is simple:\n\nDo not turn every newly extracted sentence into an immediately valid replacement fact.\n\nA plan is not an event. A correction is not always a new state. A later message can change how an earlier message should be interpreted.\n\nTemporal memory systems such as Graphiti preserve superseded facts through invalidation rather than simply deleting them, allowing current and historical relationships to remain distinguishable. [getzep](https://www.getzep.com/platform/graphiti/)\n\nBut preserving history is only part of the design. The reader must also choose the right historical state for the question.\n\nI would separate three layers in a memory implementation.\n\nWhat was actually said?\n\nKeep the source message, speaker, timestamp, and enough surrounding context to interpret it.\n\nWhat does the system think the source claims?\n\nFor example:\n\n```\n{\n  \"subject_id\": \"person_sarah_01\",\n  \"predicate\": \"employed_by\",\n  \"object_id\": \"company_acme_01\",\n  \"assertion_type\": \"reported_state\",\n  \"recorded_at\": \"2026-06-10\",\n  \"valid_from\": null,\n  \"valid_to\": null,\n  \"source_message_id\": \"message_184\"\n}\n```\n\nThis is an illustrative schema, not a complete temporal database design.\n\nThe null dates matter. If the source does not establish when employment started, the extractor should not invent a start date.\n\nWhat should the assistant currently believe?\n\nA current-state view can select the assertions relevant to “Where does Sarah work now?”\n\nA historical view can select evidence relevant to “Where did Sarah work when we met?”\n\nThese views should point back to the underlying evidence.\n\nThe architectural goal is to make interpretations revisable without losing the information used to produce them.\n\nNow introduce two people named Sarah.\n\nOne is your colleague. The other is your sister.\n\nA name alone cannot distinguish them in this example. But “name plus relationship” is not necessarily a permanent identifier either: colleagues become friends, managers change, and relationships can overlap.\n\nI would use stable internal entity IDs, with names and relationships stored as evidence-backed attributes.\n\nFor ambiguous references, the write path should have more than two options.\n\nInstead of only:\n\n```\nCreate entity\nor\nMerge into existing entity\n```\n\nallow:\n\n```\nCreate entity\nLink with sufficient evidence\nHold as unresolved\nAsk for clarification\n```\n\nA system that can preserve uncertainty does not have to turn every ambiguous mention into a confident merge.\n\nConsider four questions:\n\nThey concern the same person and topic, but they ask for different evidence.\n\nFor this design, I would represent query intent explicitly enough to distinguish:\n\nThe retrieval pipeline could then look like this:\n\n```\nQuestion\n   ↓\nEntity resolution\n   ↓\nTemporal and question-intent interpretation\n   ↓\nCandidate retrieval\n   ↓\nValidity and provenance checks\n   ↓\nEvidence selection\n   ↓\nAnswer or abstention\n```\n\nThis is a proposed decomposition, not an argument that every box requires a separate model call.\n\nThe important part is that “most similar” and “appropriate evidence” are not treated as interchangeable.\n\nSuppose a fact concerns both Sarah and Acme.\n\nIt should be discoverable through either entity.\n\nFor a structured implementation, I would prefer one canonical fact record referenced by both entity indexes rather than two independently editable copies:\n\n```\nSarah index ──┐\n              ├── fact_271\nAcme index ───┘\n```\n\nThat gives corrections one authoritative target.\n\nMaterialized copies may still be useful for performance, but then their consistency needs to be an explicit engineering responsibility.\n\nA chapter metaphor can make navigation intuitive. It should not accidentally dictate an update model that is difficult to maintain.\n\nImagine a memory system with three stages:\n\n```\nConversation → Extraction → Filing → Retrieval\n```\n\nA wrong answer could originate anywhere:\n\nImproving the retriever will not reveal which of these happened.\n\nBefore choosing the next model to train, I would run an ablation:\n\n| Condition | Extraction | Filing | \n|---|---|---|\n| A | Human-verified | Human-verified | \n| B | Model-generated | Human-verified | \n| C | Human-verified | Model-generated | \n| D | Model-generated | Model-generated | \n\nKeep the retrieval procedure fixed.\n\nCondition A is a reference condition for retrieval with verified inputs—not a theoretical upper bound.\n\nThe other conditions help investigate which upstream errors are responsible for the observed failures.\n\nThe evaluation should report those failures separately, rather than reducing every error to “memory recall.”\n\nA useful test set should contain more than questions with known, neatly stored answers.\n\nI would include:\n\nFor each case, define acceptable behavior in advance.\n\nSometimes the correct output is a fact.\n\nSometimes it is a qualified answer.\n\nSometimes it is:\n\n“I know Sarah changed employers, but I don’t have enough information to determine where she worked on that date.”\n\nAn evaluation that always requires a factual answer cannot tell you whether the system handles missing evidence appropriately.\n\nRetrieval evaluation is useful, but it does not automatically establish the performance of the complete assistant.\n\nFor a proposed system, I would track at least three groups of measurements.\n\nA selective reader may reduce context size, but that remains an empirical question until the full reading loop is integrated and measured.\n\nLikewise, an extra retrieval step might save prompt tokens while adding latency. The relevant trade-off belongs in the end-to-end results, not only in an architectural diagram.\n\nSemantic retrieval remains useful. It can help locate evidence when the user’s wording differs from the stored text.\n\nBut my design recommendation is to place it inside a broader lifecycle:\n\n```\nCapture\n→ Interpret\n→ Resolve identity\n→ Record provenance\n→ Establish validity\n→ Retrieve\n→ Verify\n→ Correct\n```\n\nExisting systems already address parts of this lifecycle. Graphiti documents temporal invalidation and historical relationships; Mem0 exposes a memory-history interface for tracking how an individual memory has changed. Neither should be reduced to “just a pile of embeddings.” [help.getzep](https://help.getzep.com/graphiti/getting-started/overview)\n\nThe useful engineering question is therefore not simply:\n\n“How do we give an assistant memory?”\n\nIt is:\n\n“What evidence does this assistant preserve, how does it revise that evidence, and under what conditions is it allowed to use it?”\n\nAn assistant that remembers yesterday’s fact is useful.\n\nAn assistant that can explain whether the fact is still true—and recognize when it cannot tell—is a more demanding system to build.", "url": "https://wpnews.pro/news/your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong", "canonical_source": "https://dev.to/ahmedadawy625/-your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong-5gbp", "published_at": "2026-10-18 21:13:12+00:00", "updated_at": "2026-10-18 21:18:53.912666+00:00", "lang": "en", "topics": ["ai-agents", "artificial-intelligence", "ai-tools"], "entities": ["Graphiti", "Zep", "Acme", "Nova", "Pune", "Sarah"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong", "markdown": "https://wpnews.pro/news/your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong.md", "text": "https://wpnews.pro/news/your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong.txt", "jsonld": "https://wpnews.pro/news/your-ai-assistant-remembered-the-fact-it-still-got-the-answer-wrong.jsonld"}}