{"slug": "ulacache-unified-logical-addressing-for-sparse-llm-serving", "title": "UlaCache: Unified Logical Addressing for Sparse LLM Serving", "summary": "A paper submitted to arXiv on 21 Oct 2026 presents UlaCache, a KV cache management system that uses a unified logical address space spanning GPU KV pools and optional CPU memory to serve sparse-attention LLM inference. With speculative decoding enabled under matched CPU and GPU memory budgets, UlaCache reached peak decoding throughput speedups of 1.16x (data-parallel) and 1.70x (tensor-parallel) on H200, and 1.73x (DP) and 2.51x (TP) on B300 versus the state-of-the-art baseline. UlaCache's Parallel Bucketed LRU (PB-LRU) updates recency metadata and selects eviction victims in parallel, and local GPU buffers retain selected KV across decoding steps so only missing entries require transfers.", "body_md": "# Computer Science > Distributed, Parallel, and Cluster Computing\n\n  [Submitted on 21 Oct 2026]\n\n# Title:UlaCache: Unified Logical Addressing for Sparse LLM Serving\n\n[View PDF](https://arxiv.org/pdf/2610.29820)\n\n[HTML (experimental)](https://arxiv.org/html/2610.29820v1)\n\nAbstract:Long contexts increase the memory cost of the key-value (KV) cache in large language model (LLM) inference. Sparse attention reduces attention computation and memory traffic by selecting historical tokens, but the retained KV cache must remain available for later decoding steps. We present UlaCache, a KV cache management system built around a unified logical address space spanning GPU KV pools and optional CPU memory. Stable logical references let the inference engine allocate and select KV independently of physical placement. Common allocation and sparse-access interfaces support data-parallel (DP) and tensor-parallel (TP) attention. For distributed storage, local GPU buffers retain selected KV across decoding steps, so only missing entries require transfers. When CPU storage is enabled, Parallel Bucketed LRU (PB-LRU) manages a GPU pool shared across requests. It updates recency metadata and selects eviction victims in parallel. UlaCache transfers KV and then updates the address mappings. With speculative decoding enabled under matched CPU and GPU memory budgets, UlaCache achieves peak decoding throughput speedups of 1.16 $\\times$ (DP) and 1.70 $\\times$ (TP) on H200, and 1.73 $\\times$ (DP) and 2.51 $\\times$ (TP) on B300, compared with the state-of-the-art baseline.\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/ulacache-unified-logical-addressing-for-sparse-llm-serving", "canonical_source": "https://arxiv.org/abs/2610.29820", "published_at": "2026-10-31 16:30:16+00:00", "updated_at": "2026-10-31 16:51:32.747719+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research", "machine-learning"], "entities": ["UlaCache", "arXiv", "H200", "B300", "Parallel Bucketed LRU"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ulacache-unified-logical-addressing-for-sparse-llm-serving", "markdown": "https://wpnews.pro/news/ulacache-unified-logical-addressing-for-sparse-llm-serving.md", "text": "https://wpnews.pro/news/ulacache-unified-logical-addressing-for-sparse-llm-serving.txt", "jsonld": "https://wpnews.pro/news/ulacache-unified-logical-addressing-for-sparse-llm-serving.jsonld"}}