{"slug": "memorilla-latent-semantic-memory-for-llms", "title": "Memorilla: Latent Semantic Memory for LLMs", "summary": "A paper posted to arXiv on 15 October 2026 (revised 3 November 2026) by Vignesh Kothapalli introduces Memorilla, a memory system that condenses an entire document collection into 16 query-dependent 'memory tokens' for a frozen LLM. Memorilla beat baselines including Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document and knowledge-intensive question answering, and reached 84.33% accuracy on FactKG versus 80.43% for full-context inference while using 108x fewer tokens. In a TextWorld environment, Memorilla raised an agent's average success rate from 18.6% with full-history in-context learning to 24.2%.", "body_md": "# Computer Science > Machine Learning\n\n  [Submitted on 15 Oct 2026 (\n\n[v1](https://arxiv.org/abs/2610.21375v1)), last revised 3 Nov 2026 (this version, v3)]\n# Title:Memorilla: Latent Semantic Memory for LLMs\n\n[View PDF](https://arxiv.org/pdf/2610.21375)\n\n[HTML (experimental)](https://arxiv.org/html/2610.21375v3)\n\nAbstract:Memory enables large language models (LLMs) to use information beyond their parameters, such as past user conversations, long documents, or an agent's earlier actions. For a frozen LLM, access to this accumulated knowledge is limited to what a memory system places in its context window. However, gathering and combining relevant information needed for each query while keeping the model's context short remains challenging for existing systems. Here we introduce Memorilla, a memory system that gathers information from an entire document collection into a fixed number of query-dependent 'memory tokens' for a frozen LLM. Memorilla encodes documents (such as user chats, book chapters) once into reusable embeddings, then learns to retrieve and place query-specific information in the LLM's input embedding space. Using only 16 memory tokens, Memorilla achieves the best performance against baselines including Mem0 and MemoRAG on five of eight benchmarks spanning personalization, long-document and knowledge-intensive question answering. Notably, on FactKG, it reaches 84.33% accuracy versus 80.43% with full-context inference, while using 108x fewer tokens to represent the source documents. Furthermore, Memorilla can facilitate continual learning in agentic systems where the document count (the agent's past turns) increases throughout the trajectory. In a TextWorld environment, we show that Memorilla reduces the agent's tendency to wander between rooms it has already explored, and improves its average success rate from 18.6% with full-history in-context learning to 24.2%. Overall, Memorilla aids frozen LLMs in accumulating and using knowledge effectively across long-context interactions and takes a step towards building continually learning AI systems.\n    \n\n## Submission history\n\nFrom: Vignesh Kothapalli [\n[view email](https://arxiv.org/show-email/bc7e86c1/2610.21375)]\n\n**Thu, 15 Oct 2026 06:55:55 UTC (3,660 KB)**\n\n[\\[v1\\]](https://arxiv.org/abs/2610.21375v1)\n**Tue, 27 Oct 2026 04:28:12 UTC (3,660 KB)**\n\n[\\[v2\\]](https://arxiv.org/abs/2610.21375v2)\n**[v3]** Tue, 3 Nov 2026 07:03:31 UTC (3,660 KB)\n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/memorilla-latent-semantic-memory-for-llms", "canonical_source": "https://arxiv.org/abs/2610.21375", "published_at": "2026-11-06 18:03:33+00:00", "updated_at": "2026-11-06 18:20:29.275168+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "machine-learning", "ai-agents", "artificial-intelligence"], "entities": ["Memorilla", "Vignesh Kothapalli", "arXiv", "Mem0", "MemoRAG", "FactKG", "TextWorld"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/memorilla-latent-semantic-memory-for-llms", "markdown": "https://wpnews.pro/news/memorilla-latent-semantic-memory-for-llms.md", "text": "https://wpnews.pro/news/memorilla-latent-semantic-memory-for-llms.txt", "jsonld": "https://wpnews.pro/news/memorilla-latent-semantic-memory-for-llms.jsonld"}}