{"slug": "the-curious-case-of-agent-memory", "title": "The Curious Case of Agent Memory", "summary": "AI agents manage memory through three approaches: context compaction, external retrieval, and learned experience, according to a developer building customer-support agents. The agent remembers by rebuilding the right context for each action, but deciding what context matters remains the core challenge. Context compaction shrinks history via deletion or summarization, though it is lossy and may omit critical details.", "body_md": "# The Curious Case of Agent Memory\n\nApproaches agents use to manage context: compaction, external retrieval, and learned experience.\n\nI have built static AI agents, where the agent has instructions and tools to fetch information (embeddings, files, etc.) from the vector database and a few other sources. It does not have to update its memory with any new information because the static information doesn't change.\n\nBut recently, I have been building this customer support AI agent where it needs to continually update its memory with information about customers, their issues, and resolutions. If Lisa (a customer support agent - real person) helped a customer with a specific issue using this AI agent, and then John (another customer support agent - real person) is trying to help another customer but with similar issues, the AI agent should know what Lisa did and suggest the same course of action to John. In short, the AI agent needs to have memory now.\n\n[The Idea](#the-idea)\n\nNow what does it mean that an AI agent has memory? Is it sentient? The conversation history is memory. A vector database is memory. A `MEMORY.md`\n\nfile is memory. Even a large context window is sometimes sold as memory. But the model does not remember a previous API call. We send it a collection of messages, tool results, and instructions, it produces an output, and that call is over. On the next call, the agent has to assemble everything again.\n\nSo where does memory come from?\n\nAn agent remembers by rebuilding the right context for its next action.\n\nDeciding what \"right\" means is the whole problem. If you give it too little, the agent has to repeat work it has already done. If you give it too much, useful information gets buried under old customer records, raw API responses, and conversations. From what I can tell, the state of the art for handling agentic memory falls into three broad approaches: compact the current context, retrieve old information from external storage, and turn past work into reusable experience.\n\n[The Context Window](#the-context-window)\n\nLet's separate the model from the agent. Every large language model has a context window. The agent is the system around it that fills that window. The customer support agent's context might look something like this:\n\n### The Context Window\n\nCRM searches, payment-provider responses, and knowledge-base results.\n\n**Key Takeaway:** Everything in this stack is sent to the LLM on\n\n*every single API call*.\n\nAll of this is sent to the model as input. If the agent loads a 2,000-line CRM export, those lines now compete with the customer's message, the support policy, and the resolution plan for attention. A one million token context window gives us more room, but it does not decide which 2,000 lines matter. It only postpones that decision.\n\nPrompt caching is useful here, but I don't think it is memory. Caching can make a repeated prefix cheaper to process. It does not choose what the agent should remember.\n\n[Approach 1: Context Compaction](#approach-1-context-compaction)\n\nThe simplest way to keep an agent running is to shrink its history when it gets too large. This is usually called compaction or context editing.\n\nSome content can be removed without much thought. If the agent looks up an order several times, it probably does not need every copy of the same record. The same goes for an early troubleshooting path that the customer has already ruled out. CRM searches and API responses take up a surprising amount of the context, so deleting stale results can buy a lot of room.\n\nEventually deletion is not enough. The agent has to summarize:\n\nThe summary becomes a smaller representation of the work so far. It is similar to compressing a log into a checkpoint and continuing from there.\n\nThis works well for the current case, but it is lossy. The summary may record that a refund was rejected and leave out the payment status that explained why. It can preserve the final decision but forget a policy constraint that led to it. Once the original context is removed, the agent cannot recover what the summary missed.\n\nThere is also a strange feedback loop here. We ask the same model that filled the context to decide which parts of that context future calls will need. Most of the time this is fine. Sometimes the model confidently throws away the one detail it should have kept.\n\n[Approach 2: External Memory and Retrieval](#approach-2-external-memory-and-retrieval)\n\nCompaction helps an agent survive one long support case. It does not help much when Lisa ends her session and John handles a similar case tomorrow. For that, the agent needs storage outside the context window.\n\nThe common solution is to save conversations, customer facts, case results, or records and retrieve a small set before each model call. The first version usually looks like this:\n\nIf John's new case is about a suspected duplicate card charge, semantic search might retrieve Lisa's earlier resolution for order #1842. This is already much better than injecting the entire support history.\n\nIt also breaks in some annoying ways. Semantic similarity is not the same as usefulness. An old duplicate-charge case may look very similar to the current one but involve a different payment provider or account. Exact order numbers and transaction IDs sometimes work better with keyword search. Recency matters. Permissions matter. And facts change.\n\nImagine the memory store contains both of these:\n\nA vector search can easily retrieve the first one. Both are relevant and semantically similar, but only one is current. Now we need timestamps, metadata, ranking, invalidation, and rules for resolving conflicts. We started with embeddings and somehow ended up rebuilding a database.\n\nNewer memory systems mix vector search with keyword search and metadata filters to handle these cases. Some use temporal knowledge graphs so that relationships can change without erasing their history. The graph can represent that one fact replaced another, instead of treating both as equally valid chunks of text.\n\nI think this is where \"agent memory\" becomes a slightly misleading term. Much of the difficult work is familiar information retrieval and database engineering. What should we write? How should we index it? How long should we retain it? Which result is authoritative? The LLM blurs those trade-offs, but it does not make them go away.\n\n[Approach 3: Learning From Experience](#approach-3-learning-from-experience)\n\nSaving and retrieving old events is useful, but an agent that remembers every event has not necessarily learned anything from them.\n\nSuppose the support agent spends 30 minutes investigating a suspected duplicate charge. It calls the payment API six times, searches the refund policy, and eventually discovers that the second charge is only a pending authorization that will reverse automatically. We could store the whole trajectory and hope a search finds it later. Or we could save one procedure:\n\nThe second version keeps the useful lesson and drops the transcript that produced it.\n\nMemory systems often borrow three categories from cognitive science:\n\n**Semantic memory** stores facts, such as a customer's contact preference or the refund policy.**Episodic memory** stores previous experiences, such as Lisa's investigation of order #1842.**Procedural memory** stores how to do something, such as the duplicate-charge resolution playbook.\n\nThe categories are useful, although real memories do not always fit cleanly into one box. A support case can contain facts about the customer, an episode about an unsuccessful resolution, and a new procedure for handling similar billing issues.\n\nAgents can form these memories while they work or in a separate reflection step. After a case, another model call examines the trajectory and asks what is worth keeping. Over time, several similar episodes can be consolidated into one general rule. A support agent might stop remembering five individual duplicate-charge cases and keep a single playbook for checking pending authorizations.\n\nThis is the newest of the three approaches, and I find it the most interesting. It is also easy to get wrong. A lesson learned from one case may not apply to a different customer, account, or payment provider. The agent can extract the wrong lesson, store it with complete confidence, and make every future case worse. Forgetting and correcting memories are as important as creating them.\n\nRecent work goes one step further and treats memory management as a policy. Instead of always saving a summary or always retrieving the nearest five chunks, the agent learns when to write, retrieve, update, and forget. Memory management then becomes another decision the agent has to make while it works.\n\n[Putting Them Together](#putting-them-together)\n\nThese approaches solve different parts of the same problem.\n\nSupport memory intuition\n\n### The support agent's desk\n\n#### A customer reports a duplicate charge\n\nLisa opens the case with the customer's message and plenty of room in the context window.\n\n#### Current case desk\n\nOnly this case context reaches the model\n\n#### Filing cabinet\n\nExternal memory\n\n#### Handbook\n\nLearned procedures\n\nCompaction keeps the current case within the context limit. Retrieval brings information back across cases and support sessions. Consolidation tries to turn the history into something more useful than a pile of transcripts.\n\nA practical support agent will probably use all three. It keeps the recent customer conversation in the prompt, clears stale CRM and API results, writes durable case notes to a database, and occasionally turns repeated resolutions into a support procedure. The exact implementation can be a sophisticated temporal graph or a collection of structured case summaries. The hard questions stay the same.\n\nWhat deserves to be remembered? When is an old memory no longer true? How does the agent know that a retrieved memory is relevant rather than merely similar? I don't think we have great answers yet.\n\nAt first, I thought a larger context window would eventually make agent memory less important. Now I am not so sure. A larger desk is useful, but at some point someone still has to clean it.", "url": "https://wpnews.pro/news/the-curious-case-of-agent-memory", "canonical_source": "https://rafihasan.com/blog/2026-07-24-the-curious-case-of-agent-memory", "published_at": "2026-07-25 18:09:40+00:00", "updated_at": "2026-07-25 18:22:25.133008+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "large-language-models"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/the-curious-case-of-agent-memory", "markdown": "https://wpnews.pro/news/the-curious-case-of-agent-memory.md", "text": "https://wpnews.pro/news/the-curious-case-of-agent-memory.txt", "jsonld": "https://wpnews.pro/news/the-curious-case-of-agent-memory.jsonld"}}