{"slug": "gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading", "title": "Gandalf: Extending SSD Lifespan for LLM KV Cache Offloading", "summary": "A paper submitted to arXiv on 22 October 2026 proposes Gandalf, an SSD KV cache offloading system for LLM serving that prioritizes full-attention KV cache admission over cache from other attention types, extending SSD lifetime by 7.3X on average with negligible impact on cache hit rate across multiple workload traces and a wide range of hardware capacities. The authors argue that gating KV cache writes purely on historical access patterns cuts SSD writes but lowers hit rates, because KV caches from different attention types can share identical past access patterns yet diverge in future ones. Gandalf is described as the first SSD KV cache offloading system with SSD lifetime awareness, and its design is said to extend to other flash-based storage such as High Bandwidth Flash (HBF).", "body_md": "# Computer Science > Machine Learning\n\n  [Submitted on 22 Oct 2026]\n\n# Title:Gandalf: Flash Shall Not Die! Increasing Flash Storage Lifetime for KV Cache Offloading in LLM Serving\n\n[View PDF](https://arxiv.org/pdf/2610.30791)\n\n[HTML (experimental)](https://arxiv.org/html/2610.30791v1)\n\nAbstract:KV caching reduces the serving cost of large language models (LLMs), but the growing KV cache footprint is increasingly exceeds GPU and host-memory capacity. Therefore, people use SSD to expand the KV cache storage capacity with high I/O bandwidth. However, in production LLM workload, such as multi-round conversation and LLM agents, the total amount of KV cache writes significantly exceeds SSD capacity, shortening SSD lifetime from years to months. To improve SSD lifetime, a natural solution is to reduce the KV cache writes by gating the KV cache writes via historical access patterns. However, that significantly lowers the KV cache hit rate (and increase the LLM serving cost) since KV caches from different attention types might have identical historical access pattern but different future access patterns. Our insight is that, to reduce KV cache writes without hurting KV cache hit rate, one need to not only consider the access pattern, but also application-level information such as the attention type of KV cache. Leveraging this observation, we propose Gandalf , the first SSD KV cache offloading system with SSD life time awareness. Gandalf prioritize the admission of full-attention KV cache into SSD over the cache generated by other types of attention, which significantly increases SSD lifetime without sacrificing cache hit rate. Our evaluation shows that Gandalf extends the SSD lifetime by 7.3X on average, while having negligible impact on cache hit rate across multiple workload traces and wide range of hardware capacity. Gandalf design can be extended to other flash-based storage, such as High Bandwidth Flash (HBF).\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading", "canonical_source": "https://arxiv.org/abs/2610.30791", "published_at": "2026-10-31 15:32:35+00:00", "updated_at": "2026-10-31 15:51:04.837341+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research", "machine-learning"], "entities": ["Gandalf", "arXiv", "High Bandwidth Flash (HBF)"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading", "markdown": "https://wpnews.pro/news/gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading.md", "text": "https://wpnews.pro/news/gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading.txt", "jsonld": "https://wpnews.pro/news/gandalf-extending-ssd-lifespan-for-llm-kv-cache-offloading.jsonld"}}