# DeepSeek V4 Flash on 8× AMD gfx1201: packaged TP=8 deployment > Source: > Published: 2026-08-20 19:57:49+00:00 Introduction DeepSeek V4 Flash on 8× AMD R9600D I *need* 256gb vram in my life. What we set out to do Eight AMD Radeon AI PRO R9600D GPUs. One 149 GB MoE model. No XGMI, no NVLink, just PCIe Gen5 and a lot of determination. DeepSeek V4 Flash is a 284B-parameter MoE with 256 routed experts, FP4 expert weights, FP8 block quantization, and a 1M-token context window. It needs eight 32gb GPUs just to fit. But can it work with 8 RDNA gpus? The answer is yes, it runs. I’m still working on the speed, though. This is a story about eight consumer-grade workstation GPUs, roughly $1,100 each, trying to do what normally requires either two $15k RTX Pro 6000 or a $30,000 datacenter accelerator. The R9600D is AMD’s RDNA4 workstation card: 48 compute units, 32 GB of VRAM, PCIe Gen5 x16. Eight of them give you 256 GB of VRAM for a fraction of the cost of something “legit datacenter.” The question turned out to be whether or not the software stack could actually make use of them. The hardware The machine is an ASRockRack GENOAD8X-2T/BCM with an AMD EPYC 9575F 64 Zen 5 cores, 128 threads, 503 GB of RAM. The eight R9600Ds are plugged directly into the CPU’s PCIe lanes, each at Gen5 x16. No PCIe switches, no XGMI, no NVLink . Eight independent paths right into the CPU’s root complex. The cards are passively cooled, which means relying on chassis airflow, and they sip about 15 watts at idle. At full tilt they’re capped at 150 W each, which is remarkably efficient for what they deliver. The model itself is DeepSeek V4 Flash, a 284B-parameter MoE with 256 routed experts, 6 active per token, 43 transformer layers, and DeepSeek’s compressed MLA attention with 1M-token context. The weights are stored in FP8 with FP4 experts about 149 GB total across 46 safetensors shards. It’s a frontier model running on hardware that costs less than a single datacenter GPU. Phase 1 RCCL ate our lunch The very first attempt at TP=8 failed before a single weight was loaded. RCCL, ROCm’s NCCL fork, would initialize fine at TP=2, TP=4, TP=6, even TP=7 but TP=8 always died with `HIP failure: invalid device pointer` . This looked like a hardware limit. It wasn’t. The clue was that TP=8 triggered a special RCCL code path called DDA (Distributed Device Allocation) IPC initialization. TP=7 didn’t. On gfx1201, that path tries to export and import GPU memory handles through HIP IPC, which the architecture doesn’t support. The fix was an upstream RCCL commit that skips DDA on unsupported architectures. We built a custom RCCL 2.30.4 with that patch, and suddenly TP=8 all-reduce passed cleanly. The environment that works is conservative: `NCCL_P2P_DISABLE=1` , `NCCL_SHM_DISABLE=0` , `NCCL_PROTO=Simple` , shared-memory transport. Native GPU P2P is asymmetric on this platform one-directional at best so we don’t use it. Shared memory over PCIe is the reliable path. Phase 2 the AITER spiral With TP=8 communication working, the model loaded. All 46 shards, 20 GiB per GPU. Then the first decode token came out, and the second one killed the worker. The fault was in AITER’s DeepGEMM paged-MQA stage-1 kernel a GPU memory fault that only appeared after the first generated token. We traced it to an ABI mismatch: vLLM was passing a page table of `[batch, max_blocks]` physical block IDs, but AITER’s kernel expected `[batch, max_tokens]` token-slot indices. The cache layout was also different vLLM stores `[num_blocks, block_size, 1, D+4]` but the gfx1201 kernel assumes a flat token-row stride. We wrote a wrapper that expands the page table to token slots, reshapes the cache to flat rows, and calls AITER’s ragged-K variant instead of the broken stage-1 kernel. It worked. Then the C4A compression boundary hit. At decoded position 2051, another kernel `_pack_global_topk_ragged_kernel` faulted with hardware exceptions on every GPU. The metadata was valid, the indices were in range, but the original 2D-grid Triton kernel crashed gfx1201. We replaced it with a simpler one-program-per-token kernel that explicitly bounds every pointer load. That fixed it. Phase 3 the Reddit clue A month-old Reddit post described the same setup: eight R9700s, DeepSeek V4 Flash, but with `VLLM_ROCM_USE_AITER=0` and `--enable-expert-parallel` . No AITER at all — just the generic Triton/Torch path. That was a fundamentally different approach from ours. We tried it. EP=8 workers initialized correctly as `Worker_TP0_EP0` through `Worker_TP7_EP7` . The model loaded. But inference failed because the generic code path called a missing DeepGEMM backend for the paged-MQA logits kernel. We added a fallback routing those calls through the existing ROCm Torch reference implementations, and it worked. The numbers After all this, the results are: | Configuration | PP2048/TG8192 | | AITER-on, corrected GPU MQA path | **3.91 tok/s** | | AITER-off + EP=8, Torch fallback | **3.89 tok/s** | Two completely different code paths, same answer: about four tokens per second. That’s roughly 250 milliseconds per token — an eternity in GPU time. What’s still wrong Four tokens per second is not what this hardware can do. These GPUs have 1,080 MHz clocks, 256-bit GDDR6 at 1,258 MHz, and 48 compute units each. We can run Qwen3.6-27B at hundreds of tokens per second on this same system — the R9600D is a capable inference card. The bottleneck is specific to TP=8 with DeepSeek V4. The leading theory is that each decode token pays a large fixed cost for TP=8 shared-memory collectives across eight GPUs — 43 layers times multiple all-reduces per layer, all over PCIe without XGMI. The DGX Spark comparison (~40 tok/s) uses two machines connected by 200 Gb InfiniBand, which is a fundamentally different communication topology. On our system, every all-reduce traverses the host memory fabric rather than a dedicated GPU interconnect. With 43 layers and at least two collectives per layer, that adds up to roughly 86 synchronization points per token. Even at modest latency per collective, the overhead dominates the ~250 ms decode budget. There are also known missing pieces. The gfx1201 FP8 kernel tuning configurations are absent — vLLM logs warnings for every GEMM shape that lacks a tuned config. The MQA fallback is still the slow Torch reference implementation rather than a fused Triton kernel. The latest upstream vLLM source (July 16) has improvements to the generic ROCm sparse-indexer path and better FP8 handling, but we haven’t been able to deploy it because the TheRock base image and the new vLLM source have conflicting dependency versions. We built it successfully — it just won’t start DeepSeek V4 in the mismatched container. What’s next The immediate path forward is a matched newer vLLM nightly container that can run the generic AITER-off path with proper Triton MQA kernels instead of the Torch fallback. The latest upstream source builds fine — we just need it in a compatible base image. That, plus generating gfx1201 FP8 tuning data for the observed GEMM shapes, should close the gap significantly. We also want to investigate whether expert parallelism actually reduces per-token collective overhead on this platform, or whether the TP=8 communication cost is simply the dominant term regardless of EP configuration. The Reddit result suggested EP helps, but our measurements showed no improvement — possibly because our Torch MQA fallback masked any EP benefit. The bottom line DeepSeek V4 Flash runs correctly on eight R9600Ds. TP=8 works, the model generates coherent output, and the full 8,192-token generation completes without crashing. The speed is not where it needs to be yet — roughly 4 tok/s versus a target in the tens — but every individual piece has been made functional one kernel at a time. And critically: **this is a DeepSeek V4 TP=8 problem, not a GPU problem.** These same cards run Qwen3.6-27B at hundreds of tokens per second without breaking a sweat. The R9600D is fast — it’s the TP=8 collective overhead and the immature gfx1201 kernel ecosystem that are the bottlenecks. As the ROCm and vLLM stacks mature for RDNA4, those numbers will climb. The hardware is ready. The software is catching up.