{"slug": "tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate", "title": "TensorRT-LLM adds Mooncake cache sharing to its v1.4 release candidate", "summary": "NVIDIA's TensorRT-LLM v1.4.0rc1 release candidate, published October 22nd, lists an integration with Mooncake Store that lets serving instances share KV-cache prefixes, with Mooncake's project update dating the merge to October 10th. The connector trades repeated prompt computation for shared host-memory capacity and KV-transfer overhead, and operators must weigh those costs against the GPU KV capacity still needed to serve cache hits.", "body_md": "# TensorRT-LLM adds Mooncake cache sharing to its v1.4 release candidate\n\n**NVIDIA's October 22nd release candidate lists the connector, which lets serving instances reuse cached prompt prefixes. Operators trade repeated prompt computation for shared host-memory capacity and KV-transfer overhead.**\n\n        By [Ryan Merket](https://runtimewire.com/author/ryan-merket)\n        · Published \n\nPrimary source: [The Cloudflare Blog](https://blog.cloudflare.com/kv-cache-exposure-remediation/)\n\n## Why it matters\n\nShared prefix reuse can cut repeated prompt computation when requests arrive at different serving instances. Operators must weigh that potential saving against host-memory reservations, transfer copies and the GPU KV capacity still required to serve cache hits.\n\nTensorRT-LLM's [v1.4.0rc1 release candidate](https://github.com/NVIDIA/TensorRT-LLM/releases?ref=runtimewire), released October 22nd, lists an integration with Mooncake Store for sharing KV-cache prefixes across serving instances. Mooncake's [project update](https://kvcache-ai.github.io/Mooncake/?ref=runtimewire) dates the merge to October 10th.…\n\nWhen you purchase through links in our articles, [we may earn a small commission](https://runtimewire.com/affiliate-monetization-standards). This doesn’t affect our editorial independence.", "url": "https://wpnews.pro/news/tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate", "canonical_source": "https://runtimewire.com/article/cloudflare-glm-53-kv-cache-exposure-155-responses", "published_at": "2026-10-27 04:29:34+00:00", "updated_at": "2026-10-27 07:42:56.051931+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools", "developer-tools"], "entities": ["NVIDIA", "TensorRT-LLM", "Mooncake Store", "Mooncake"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate", "markdown": "https://wpnews.pro/news/tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate.md", "text": "https://wpnews.pro/news/tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate.txt", "jsonld": "https://wpnews.pro/news/tensorrt-llm-adds-mooncake-cache-sharing-to-its-v1-4-release-candidate.jsonld"}}