Supermicro, IBM and NVIDIA put KV cache on shared storage and claim 56x faster first tokens

Supermicro announced on November 5th, 2026 a KV cache solution built with IBM and NVIDIA that the companies say delivered up to 56x faster first tokens in joint testing with reused long context at 130,000-token prompts. The system combines Supermicro Petascale all-flash storage servers, IBM Storage Scale software, NVIDIA Dynamo inference software and NVIDIA Spectrum-X Ethernet networking to persist KV cache on shared storage, letting operators reuse context across requests and nodes instead of rebuilding it on GPUs. Supermicro founder Charles Liang is pushing the company deeper into the inference stack, targeting long-context and agentic workloads where repeated context rebuilds consume GPU cycles before the first token appears.

Supermicro, IBM and NVIDIA put KV cache on shared storage and claim 56x faster first tokens Supermicro, IBM and NVIDIA say persistent KV cache on Petascale storage delivered up to 56x faster first tokens when long context was reused at 130,000-token prompts in joint testing. By RuntimeWire Staff https://runtimewire.com/author/runtimewire-staff · Published · Updated Primary source: PR Newswire Business Technology https://www.prnewswire.com/news-releases/supermicro-launches-kv-cache-solution-with-ibm-and-nvidia-demonstrating-significant-inference-performance-benefits-302928472.html Why it matters Long-context and agentic workloads can require models to rebuild the same context again and again, using GPU cycles before the first token appears. A persistent, shared KV cache tier gives operators a way to reuse that work across requests and nodes, which can raise GPU utilization and concurrency without buying GPUs in proportion to context growth. Supermicro founder Charles Liang @charlesliang https://x.com/charlesliang?ref=runtimewire is pushing his company deeper into the inference stack, this time with a storage system built to stop GPUs from redoing work they have already done. On November 5th, 2026, Supermicro announced a KV cache solution with IBM and NVIDIA that combines Supermicro Petascale all-flash storage servers, IBM Storage Scale software, NVIDIA Dynamo inference software and NVIDIA Spectrum-X Ethernet networking.… When you purchase through links in our articles, we may earn a small commission https://runtimewire.com/affiliate-monetization-standards . This doesn’t affect our editorial independence.