cd /news/artificial-intelligence/nvidia-s-tensorrt-llm-more-than-doub… · home › topics › artificial-intelligence › article
[ARTICLE · art-170677] src=runtimewire.com ↗ pub= topic=artificial-intelligence verified=true sentiment=↑ positive

NVIDIA's TensorRT-LLM more than doubles DeepSeek cache capacity in one test

NVIDIA merged pull request #21313 into TensorRT-LLM main on November 16th, 2026, changing how the dense NVFP4 multi-head latent attention cache is sized and reused for DeepSeek-V3/R1-style models. In contributor Qi Zhang's DeepSeek-R1 serving run with multi-token prediction, a context worker with 43 GiB reserved for KV cache gave its MLA layers 16.1 GiB (787,000 tokens) under the old allocator and 38.9 GiB (1.90 million tokens) under the new batch-based allocator. The change more than doubles usable cache capacity for DeepSeek-style models without increasing the 43 GiB reservation.

read1 min views6 publishedNov 16, 2026
NVIDIA's TensorRT-LLM more than doubles DeepSeek cache capacity in one test
Image: Runtimewire (auto-discovered)

NVIDIA's batch-based allocator gave MLA layers 38.9 GiB of a 43 GiB DeepSeek-R1 cache reservation, up from 16.1 GiB in one reported layout.

        By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)
        · Published 

NVIDIA merged pull request #21313 into TensorRT-LLM main on November 16th, 2026, changing how the dense NVFP4 multi-head latent attention cache is sized and reused for DeepSeek-V3/R1-style models. In contributor Qi Zhang's DeepSeek-R1 serving run with multi-token prediction, a context worker with 43 GiB reserved for KV cache previously gave its MLA layers 16.1 GiB, enough for 787,000 tokens. The new allocator gave those layers 38.9 GiB and 1.90 million tokens. The pull request is credited to the GitHub account Tracin, and commit signatures in the merged history identify Qi Zhang.…

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @nvidia 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/nvidia-s-tensorrt-ll…] indexed:0 read:1min 2026-11-16 · —