Supermicro, IBM and NVIDIA say persistent KV cache on Petascale storage delivered up to 56x faster first tokens when long context was reused at 130,000-token prompts in joint testing.
By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)
· Published
· Updated
Primary source: PR Newswire Business Technology
Why it matters #
Long-context and agentic workloads can require models to rebuild the same context again and again, using GPU cycles before the first token appears. A persistent, shared KV cache tier gives operators a way to reuse that work across requests and nodes, which can raise GPU utilization and concurrency without buying GPUs in proportion to context growth.
Supermicro founder Charles Liang (@charlesliang) is pushing his company deeper into the inference stack, this time with a storage system built to stop GPUs from redoing work they have already done. On November 5th, 2026, Supermicro announced a KV cache solution with IBM and NVIDIA that combines Supermicro Petascale all-flash storage servers, IBM Storage Scale software, NVIDIA Dynamo inference software and NVIDIA Spectrum-X Ethernet networking.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.