{"slug": "vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x", "title": "vLLM cuts a DeepSeek V4.1 Engram lookup from 54 ms to 5.1 ms on eight MI355X GPUs", "summary": "An AMD contributor filing as Fangzhou Ai merged a vLLM change that cut a DeepSeek V4.1 Engram lookup from 54 ms to 5.1 ms, a 10.6x speedup, on eight MI355X GPUs. The change covers a 589,824-row lookup against a 94 GiB shared table and traces the prefill stall to the order in which GPUs fetched rows from a host-resident table, ordering those requests before they reached host memory. The sort path is enabled on ROCm from 32,768 rows.", "body_md": "# vLLM cuts a DeepSeek V4.1 Engram lookup from 54 ms to 5.1 ms on eight MI355X GPUs\n\n**Contributor Fangzhou Ai's 10.6x result covers a 589,824-row lookup against a 94 GiB shared table on eight MI355X GPUs, with the sort path enabled on ROCm from 32,768 rows.**\n\n        By [RuntimeWire Staff](https://runtimewire.com/author/runtimewire-staff)\n        · Published \n\nFangzhou Ai, an AMD contributor filing as [\\[email protected\\]](https://runtimewire.com/cdn-cgi/l/email-protection#ec8a8d828b96848399c28d85ac8d8188c28f8381), traced a DeepSeek V4.1 prefill stall on MI355X to the order in which GPUs fetched rows from a host-resident table. The merged vLLM change cut the largest tested lookup from 54 ms to 5.1 ms by ordering those requests before they reached host memory.…", "url": "https://wpnews.pro/news/vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x", "canonical_source": "https://runtimewire.com/article/vllm-deepseek-v41-engram-lookup-mi355x", "published_at": "2026-11-16 03:12:08+00:00", "updated_at": "2026-11-16 03:21:51.384227+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-chips", "mlops"], "entities": ["vLLM", "DeepSeek V4.1", "Fangzhou Ai", "AMD", "MI355X", "ROCm"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x", "markdown": "https://wpnews.pro/news/vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x.md", "text": "https://wpnews.pro/news/vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x.txt", "jsonld": "https://wpnews.pro/news/vllm-cuts-a-deepseek-v4-1-engram-lookup-from-54-ms-to-5-1-ms-on-eight-mi355x.jsonld"}}