{"slug": "your-ai-agent-doesn-t-know-when-its-memory-is-gone", "title": "Your AI agent doesn't know when its memory is gone", "summary": "Researchers introduced MemDecay, a training-free KV-cache eviction policy that uses semantic region awareness to manage memory in LLM agents, outperforming recency-based methods by preserving critical system tokens with near-perfect accuracy. The method assigns region-specific decay rates and pinning, addressing the memory bottleneck from heterogeneous agent contexts, though attention-score normalization remains a limitation.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 12 Jul 2026]\n\n# Title:MemDecay: Region-Aware KV Cache Eviction for Efficient LLM Agent Inference\n\n[View PDF](/pdf/2607.10582)\n\n[HTML (experimental)](https://arxiv.org/html/2607.10582v1)\n\nAbstract:Large language model (LLM) agents accumulate heterogeneous context, including system instructions, plans, user turns, retrieved documents, tool outputs, and intermediate reasoning, whose key-value (KV) cache can become a major memory bottleneck. Existing eviction policies generally apply the same attention- or recency-based rule to every token, ignoring semantic structure already available to the agent orchestrator.\n\nWe introduce MemDecay, a training-free, region-aware KV-cache eviction policy. MemDecay assigns tokens region-specific base priorities and decay rates, refreshes retention scores when tokens receive attention, and evicts the lowest-scoring pages under a fixed cache budget while allowing critical regions to be pinned. We also provide a procedure for calibrating decay rates from measured attention lifetimes.\n\nWe evaluate MemDecay at approximately 450 and 1,700 token contexts using Qwen2.5-1.5B and 3B. Across all settings, attention lifetimes differ by an order of magnitude across regions: system-token half-lives range from 148 to 189 decoding steps, compared with 14 to 16 for scratchpad tokens. Pinning preserves system-region facts at full-cache accuracy in every setting, while no baseline preserves more than 13 of 24. Region-aware retention remains effective as context grows, whereas recency-based retention collapses. Accumulated-attention retention performs better on unpinned content, however, and ablations identify attention-score normalization as the main limitation of the current formulation. These results establish semantic prompt structure as a robust signal for KV-cache management while clarifying how it should be combined with attention-based importance.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/your-ai-agent-doesn-t-know-when-its-memory-is-gone", "canonical_source": "https://arxiv.org/abs/2607.10582", "published_at": "2026-07-16 21:37:24+00:00", "updated_at": "2026-07-16 21:55:16.468369+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research"], "entities": ["MemDecay", "Qwen2.5-1.5B", "Qwen2.5-3B"], "alternates": {"html": "https://wpnews.pro/news/your-ai-agent-doesn-t-know-when-its-memory-is-gone", "markdown": "https://wpnews.pro/news/your-ai-agent-doesn-t-know-when-its-memory-is-gone.md", "text": "https://wpnews.pro/news/your-ai-agent-doesn-t-know-when-its-memory-is-gone.txt", "jsonld": "https://wpnews.pro/news/your-ai-agent-doesn-t-know-when-its-memory-is-gone.jsonld"}}