Why KV Cache Offload Changes the Storage Game #
KV cache is the memory bottleneck everyone hits when running long-context LLM inference. The transformer holds key-value tensors for every token, and with contexts stretching past 128K tokens, that state either fits in HBM or becomes a serious problem. Nvidia's CMX architecture seems designed to push that cache out of the GPU memory pool, which means the storage tier has to behave more like memory than a typical NVMe drive.
That's exactly where PCIe 6.0 matters. The protocol doubles bandwidth per lane compared to PCIe 5.0 — 64 GT/s instead of 32 GT/s. For an x4 link that's already significant, but for the CM10, the whole point is that you can feed a cache-hungry accelerator without making the I/O path the new bottleneck.
What the CM10 Actually Brings #
Kioxia made some choices here that are worth noting:
PCIe 6.0 interface with support for the new PAM4 signaling, which is a big leap over NRZ. It's not just marketing — it halves latency per unit of bandwidth.Enterprise durability tier, meaning it's built for sustained writes, not just benchmark reads. KV cache workloads are write-heavy during prompt processing and read-heavy during token generation.A form factor that likely fits standard E3.S or E1.S slots, so you don't need custom backplanes to test it.** Targeting Nvidia CMX specifically**, which is a strong signal that Kioxia sees this as an accelerator sidecar, not a general-purpose disk replacement.
I don't have the full datasheet in front of me, so I won't quote IOPS or latency numbers I can't verify. The interesting part is the positioning. Instead of selling raw capacity, Kioxia is selling a low-latency, high-bandwidth cache tier that sits between DRAM and conventional storage.
The Catch with PCIe 6.0 SSDs #
Real talk: PCIe 6.0 controllers are expensive, and the NAND to saturate even half of that bandwidth isn't cheap either. Most existing flash memory can do maybe 10–15 GB/s per device, while PCIe 6.0 x4 links can theoretically carry over 60 GB/s. So the CM10 is not about maxing out the interface — it's
Next Thomson Reuters' In-House AI Model Ranks Among the Best →