{"slug": "doc-to-atom-learning-to-compile-and-compose-memory-atoms", "title": "Doc-to-Atom: Learning to Compile and Compose Memory Atoms", "summary": "Researchers have developed Doc-to-Atom (Doc2Atom), a compositional parametric memory framework that decomposes documents into typed knowledge atoms compiled into independent micro-LoRA adapters. The system uses a query router to select and assemble only relevant atoms into query-specific adapters, addressing interference and scalability issues in prior context distillation methods. Experiments on six QA benchmarks show Doc2Atom outperforms Doc-to-LoRA baselines while reducing the memory cost of document internalization.", "body_md": "# Computer Science > Computation and Language\n\n[Submitted on 10 Jun 2026]\n\n# Title:Doc-to-Atom: Learning to Compile and Compose Memory Atoms\n\n[View PDF](/pdf/2606.12400)\n\n[HTML (experimental)](https://arxiv.org/html/2606.12400v1)\n\nAbstract:Long input sequences are central to document understanding and multi-step reasoning in Large Language Models, yet the quadratic cost of attention makes inference both memory-intensive and slow. Context distillation mitigates this by compressing contextual information into model parameters, and recent work such as Doc-to-LoRA amortizes context distillation into a single forward pass that generates one LoRA adapter per document. However, producing a single monolithic adapter for all queries leads to irrelevant-query interference, limited compositional recall, and poor scalability to long-document reasoning. To address these challenges, we propose Doc-to-Atom (Doc2Atom), a compositional parametric memory framework that decomposes each document into semantically typed knowledge atoms. Each atom is compiled into an independent micro-LoRA adapter and a provenance retrieval key. At inference time, a lightweight query router selects and assembles only the relevant atoms into a query-specific adapter, which is then injected into a frozen base model. The entire system is trained end-to-end through a multi-objective distillation framework. Experiments on six diverse QA benchmarks demonstrate that Doc2Atom outperforms Doc-to-LoRA baselines while reducing the memory cost of document internalization.\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/doc-to-atom-learning-to-compile-and-compose-memory-atoms", "canonical_source": "https://arxiv.org/abs/2606.12400", "published_at": "2026-06-12 01:00:22+00:00", "updated_at": "2026-06-12 01:47:47.386839+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "artificial-intelligence", "natural-language-processing", "ai-research"], "entities": ["Doc-to-Atom", "Doc2Atom", "Doc-to-LoRA", "LoRA"], "alternates": {"html": "https://wpnews.pro/news/doc-to-atom-learning-to-compile-and-compose-memory-atoms", "markdown": "https://wpnews.pro/news/doc-to-atom-learning-to-compile-and-compose-memory-atoms.md", "text": "https://wpnews.pro/news/doc-to-atom-learning-to-compile-and-compose-memory-atoms.txt", "jsonld": "https://wpnews.pro/news/doc-to-atom-learning-to-compile-and-compose-memory-atoms.jsonld"}}