{"slug": "dapustor-s-kv-cache-nvme-ssd-offload-explainer", "title": "DapuStor's KV Cache NVMe SSD offload explainer", "summary": "DapuStor published a 5-page explainer document showing how tiering evicted KV cache contents to NVMe SSDs can solve the recomputation delay problem in LLM inference, which is limited first by GPU HBM capacity and then by tiered caching through x86 host DRAM to internal and external NVMe SSDs. The document describes how vLLM and LMCache pack KV cache data from multiple tokens for transfer, letting external storage generate flash-friendly sequential large-block I/O patterns that improve bandwidth utilization on NVMe SSDs, illustrated with diagrams and Python code examples. DapuStor notes PCIe Gen5 NVMe SSD capacities reaching 245 TB, with more expected next year, leaving scope for petabyte-level external KV cache storage.", "body_md": "# DapuStor's KV Cache NVMe SSD offload explainer\n\nAI inferencing is memory-limited, bottlenecked by [KV Cache](https://www.blocksandfiles.com/ai-ml/2026/03/30/nvidia-and-its-partners-kv-cache-extenders/5209284) capacity limits dictated by a GPU's high-bandwidth memory (HBM) amount and then by the tiered KV caching scheme via the x86 hosts DRAM, through to internal and external NVMe SSDs. We have discussed KV cache news and developments many times and [DapuStor](https://www.blocksandfiles.com/flash/2026/09/21/a-look-at-dapustors-combined-slc-and-qlc-ssd/5297708) has put together a 5-page explainer document that shows how tiering evicted KV cache contents to NVMe SSDs can solve the recomputation delay problem.\n\nThe document says offloading large KV Cache datasets to a more cost-effective external storage tier is emerging as a practical approach for scaling inference clusters. It neatly explains how KV Cache works in LLM inference and discusses how NVMe SSDs can help build a cost-efficient storage layer for AI infrastructure.\n\nThe document discusses the mechanisms used by [vLLM](https://github.com/vllm-project/vllm) and [LMCache](https://lmcache.ai/en/) to make SSD-based KV Cache more effective. These are explained with text, diagrams and a couple of Python code examples. For example;\n\nIt says \"Both vLLM and LMCache support packing KV Cache data from multiple tokens for transfer, improving memory efficiency and I/O efficiency.\"\n\nThe point here is that \"External storage systems can use these larger chunks to generate flash-friendly, sequential large-block I/O patterns, improving bandwidth utilization on NVMe SSDs.\"\n\nWith PCIe gen5 NVMe SSDs having capacities rise to 245 TB, and hopefully beyond that next year, there is a lot of scope for building petabyte-level external storage KV caches with good bandwidth and high capacity. Our version of DapuStor's paper, generalized to all NVMe SSDs, can be downloaded here.", "url": "https://wpnews.pro/news/dapustor-s-kv-cache-nvme-ssd-offload-explainer", "canonical_source": "https://www.blocksandfiles.com/flash/2026/10/26/dapustors-kv-cache-nvme-ssd-offload-explainer/5305417", "published_at": "2026-10-26 15:09:06+00:00", "updated_at": "2026-10-26 15:42:53.603344+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools"], "entities": ["DapuStor", "vLLM", "LMCache", "Blocks & Files"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/dapustor-s-kv-cache-nvme-ssd-offload-explainer", "markdown": "https://wpnews.pro/news/dapustor-s-kv-cache-nvme-ssd-offload-explainer.md", "text": "https://wpnews.pro/news/dapustor-s-kv-cache-nvme-ssd-offload-explainer.txt", "jsonld": "https://wpnews.pro/news/dapustor-s-kv-cache-nvme-ssd-offload-explainer.jsonld"}}