Stanford researchers compress LLM memory into 16 tokens and beat full context on five benchmarks

Adibvafa Fallahpour and six co-authors at Stanford released Memorilla on November 5th, a research system that compresses what a frozen large language model must remember into 16 query-dependent memory tokens, scoring 44.6% on PersonaMem-v2 versus 30.9% for the next best method. Fallahpour described the approach as "memory as a modality for LLMs," with the tokens fed into the model in the same position image embeddings occupy in a multimodal system. The authors' results show learned memory connects weakly related facts better than RAG but loses exact names that RAG keeps verbatim, pointing to hybrid systems rather than a replacement.

Stanford researchers compress LLM memory into 16 tokens and beat full context on five benchmarks Fallahpour and colleagues at Stanford report 44.6% on PersonaMem-v2 against 30.9% for the next best method, using 16 tokens instead of the full collection. By Ryan Merket https://runtimewire.com/author/ryan-merket · Published Primary source: arXiv https://arxiv.org/abs/2610.21375 Why it matters Memory is where personalization and agent systems currently pay full-context prices or accept retrieval misses. A trainable module that reads an entire collection and answers in 16 tokens would cut inference cost sharply if it transfers beyond Qwen models and benchmark collections. The authors' own results show the trade: learned memory connects weakly related facts better than RAG and loses exact names that RAG keeps verbatim, which points to hybrid systems rather than a replacement. Adibvafa Fallahpour @adibvafa https://x.com/adibvafa and six co-authors released Memorilla on November 5th, a research system that compresses everything a frozen large language model needs to remember into 16 query-dependent memory tokens. In a thread on X https://x.com/adibvafa/status/2118434330590761398 , Fallahpour described the approach as "memory as a modality for LLMs," with the tokens fed into the model in the same position image embeddings occupy in a multimodal system. When you purchase through links in our articles, we may earn a small commission https://runtimewire.com/affiliate-monetization-standards . This doesn’t affect our editorial independence.