{"slug": "the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation", "title": "The sm_120 shared-memory cliff: why FP8 KV cache crashes vLLM on workstation Blackwell", "summary": "A developer investigating a vLLM crash on workstation Blackwell GPUs found that FP8 KV cache kernels exceed the per-block shared-memory limit on sm_120 hardware, causing OutOfResources errors. The issue stems from a 1,024-byte gap between the per-SM and per-block budgets, and the developer provides a diagnostic method and a proposed fix that checks the actual device limit.", "body_md": "If you run vLLM with `--kv-cache-dtype fp8`\n\non a DeepSeek-family (MLA) model and your GPU is a GB10, an RTX PRO 6000, or any workstation or consumer Blackwell card, there is a decent chance the engine dies on startup with:\n\n```\ntriton.runtime.errors.OutOfResources: out of resource: shared memory,\nRequired: 102400, Hardware limit: 101376. Reducing block sizes or `num_stages` may help.\n```\n\nWe hit this, measured exactly where the 1,024 missing bytes come from, and posted the measurements on the upstream issue (vllm-project/vllm#53748). This post is the diagnostic method, because it applies to any Triton kernel on this hardware class, not just this one bug.\n\nDatacenter Blackwell parts advertise a larger per-block shared-memory budget than the workstation and consumer silicon. On an RTX PRO 6000 Blackwell (sm_120), `torch.cuda.get_device_properties`\n\nreports:\n\n| property | bytes |\n|---|---|\n`shared_memory_per_block_optin` |\n101,376 |\n`shared_memory_per_multiprocessor` |\n102,400 |\n\nGB10 (DGX Spark) reports the same 101,376-byte per-block opt-in limit. So a kernel tuned to fit 102,400 bytes fits the SM but not the block, and the whole workstation Blackwell class fails together. If your kernel config was validated on H100 or B200 numbers, this is the cliff you fall off when someone runs it on a desk machine.\n\nDo not debug this inside the serving engine. Launch the suspect kernel directly with the exact tile configuration the engine would pin, sweep `num_stages`\n\n, and read the compiled kernel's shared-memory size. For vLLM's MLA decode kernel (`_fwd_grouped_kernel_stage1`\n\n) with the DeepSeek tile shape (`BLOCK_DMODEL=512, BLOCK_DPE=64, BLOCK_DV=512, BLOCK_N=32, BLOCK_H=16, num_warps=4`\n\n):\n\n| num_stages | KV bf16 | KV fp8e4m3 |\n|---|---|---|\n| 1 | 59,392 (launches) | 83,968 (launches) |\n| 2 | 63,488 (launches) | 102,400 (OutOfResources) |\n| 3 | 63,616 (launches) | 102,656 (OutOfResources) |\n| 4 | 63,616 (launches) | 102,656 (OutOfResources) |\n\nThe table says everything. With bf16 KV, an extra pipeline stage costs about 4 KB and even four stages sit far under the limit. With fp8 KV, a stage costs about 18 KB, and stage two lands on 102,400 bytes: exactly the full per-SM budget, and 1,024 bytes over the per-block cap. The crash is specific to the fp8 KV path; MLA with bf16 KV is fine at these tile sizes.\n\nThe probe is ordinary Triton: import the kernel function, build dummy tensors for the tile shape, launch once per (dtype, num_stages) combination, and catch `OutOfResources`\n\n. The compiled size is on `kernel.metadata`\n\nafter a successful launch. No server, no model download, under a minute per configuration.\n\nDeepSeek-V2-Lite-Chat is the smallest convenient MLA model with the same `Lk=576`\n\nhead layout (512 latent plus 64 rope):\n\n```\nvllm serve deepseek-ai/DeepSeek-V2-Lite-Chat --trust-remote-code \\\n  --enforce-eager --kv-cache-dtype fp8 --max-model-len 4096\n```\n\nOn sm_120 the backend selector reports `TRITON_MLA`\n\nas the only MLA decode option, so there is no fallback path: the engine dies during the startup dummy run (vLLM 0.27.1, torch 2.13.0+cu130, triton 3.7.1, CUDA 13.1).\n\nThe upstream patch direction pins `num_stages=1`\n\nfor MLA kernels on non-HIP devices, which makes the server start and generate correctly (we validated greedy output on sm_120 after applying it). But the sweep shows why a blanket `is_mla`\n\ncondition costs something: bf16 KV at two stages fits comfortably (63,488 < 101,376), and the second stage exists for throughput. The robust rule is to compute the tile's actual shared-memory requirement and compare it against `shared_memory_per_block_optin`\n\nfor the device you are actually on. That handles GB10, RTX PRO 6000, and whatever ships next, without a special case per chip.\n\nThe general lesson for anyone shipping Triton kernels: your shared-memory budget is a per-block, per-device property, not a compile-time constant, and the workstation cards your users actually own report smaller numbers than the datacenter parts your CI runs on. Probe first; it takes five minutes.\n\nThis came out of inference engineering work on Blackwell workstations. The benchmark that surrounds it, with raw data and reproduction scripts, is here: [https://conatus.jahn.ai/ai-engineering/sample-report](https://conatus.jahn.ai/ai-engineering/sample-report)", "url": "https://wpnews.pro/news/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation", "canonical_source": "https://dev.to/conatusai/the-sm120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation-blackwell-1dji", "published_at": "2026-08-25 21:02:33+00:00", "updated_at": "2026-08-25 21:14:17.241869+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "developer-tools"], "entities": ["vLLM", "Triton", "DeepSeek", "NVIDIA", "GB10", "RTX PRO 6000"], "alternates": {"html": "https://wpnews.pro/news/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation", "markdown": "https://wpnews.pro/news/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation.md", "text": "https://wpnews.pro/news/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation.txt", "jsonld": "https://wpnews.pro/news/the-sm-120-shared-memory-cliff-why-fp8-kv-cache-crashes-vllm-on-workstation.jsonld"}}