{"slug": "native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16", "title": "Native vLLM and ROCm 7.15 for RX 6000 (RDNA2) on Windows 11 – 26 Tflops FP16", "summary": "A new native installer brings vLLM and ROCm 7.15 (TheRock) to AMD Radeon RX 6000 (RDNA2) GPUs on Windows 11, achieving 26 TFLOPS FP16 and 58-62 tok/s on Qwen3.5-4B, verified on an RX 6750 XT 12 GB. The one-click installer, developed by sebastianmechno-sys, eliminates WSL2, NVIDIA, and compiler requirements, providing an OpenAI-compatible chat server and prebuilt binaries.", "body_md": "Native vLLM + ROCm 7.15 (TheRock) for the whole AMD Radeon RDNA2 family on\nWindows — **no WSL2, no NVIDIA, no compiler**. One-click installer, everything\nprebuilt, and an OpenAI-compatible chat server that looks and works like the\nNVIDIA stack.\n\n**Verified on AMD Radeon RX 6750 XT 12 GB (gfx1031) — Windows 11 native — August 2026**\n\n| Result | Number |\n|---|---|\n| rocBLAS FP16 GEMM (native bench) | 25 674 Gflops ≈ 26 TFLOPS |\n| vLLM decode, Qwen3.5-4B 4-bit | ~58-62 tok/s (8.3 → 62.5 = 7.5× optimized) |\n\n| Item | Requirement |\n|---|---|\n| OS | Windows 10/11 (Windows 11 recommended; `tar` must support zstd — automatic on Win11) |\n| GPU | AMD RDNA2 — RX 6400 / 6500 / 6600 / 6650 / 6700 / 6750 / 6800 / 6900 / 6950 (all XT/M variants), 8+ GB VRAM for the 4B model |\n| Driver | AMD Software: Adrenalin Edition (the normal gaming driver) |\n| Disk | ~25 GB free on `C:` |\n| Internet | only during install (~6 GB: stack ~2.3 GB + model ~3.8 GB) |\n| Admin | one UAC click (installer auto-elevates) |\n\nNo compiler, no ROCm installer, no manual setup — everything ships prebuilt.\n\n-\nDownload the repository (ZIP or\n\n`git clone`\n\n). You do**not** need to download the release archives (`*.tar.zst`\n\n) manually — the installer fetches them automatically from the[Releases](/sebastianmechno-sys/vllm-rocm-windows-rdna2/releases)tab. -\n**Double-click**— by default it downloads from this repo's releases; pass a GitHub username to use your own fork instead. It checks GPU + disk, then installs everything it does not already have (re-run is always safe and fast):`INSTALL.bat`\n\nStep Action 1/6 GPU detection (warns if not RDNA2) + Python 3.11.9 2/6 4 archives from GitHub Releases → `C:\\Python311`\n\n,`C:\\TheRock`\n\n,`C:\\vw_*_build`\n\n3/6 venv fix + torch self-check 4/6 Qwen3.5-4B 4-bit model (skipped if already in HuggingFace cache) 5/6 writes `config.bat`\n\n(the file to edit to change model later)6/6 verification benchmark -\n→ opens the web chat in your browser (starts the model server automatically the first time). While the model thinks you see a small`CHAT.bat`\n\n*Thinking…*spinner; when it's done, only the final answer streams in — the internal reasoning stays hidden. Tokens stream live with a tok/s counter, everything local on your AMD GPU. -\n→ starts the model server on its own (OpenAI-compatible API on`SERVE.bat`\n\n`http://127.0.0.1:8000/v1`\n\n, like`vllm serve`\n\non NVIDIA). Use it with any OpenAI client, or just run`CHAT.bat`\n\n. -\n→ all 3 verification checks in one run: ROCm GPU detection, native rocBLAS FP16 power (`VERIFY.bat`\n\n**~26 TFLOPS**) and the full 512-token vLLM benchmark (**~58-62 tok/s**).\n\n| Test | Config | Result |\n|---|---|---|\n| rocBLAS FP16 GEMM | 4096×4096×4096, rocblas-bench.exe | 25 674 Gflops (≈26 TFLOPS) |\n| vLLM Qwen3.5-4B decode | 512 tok, greedy, CUDA graphs | 59.4 tok/s (up to 62.5) |\n| Optimization progression | eager fp16 baseline | 8.3 → 62.5 tok/s (7.5×) |\n\nFull optimization history:\n\n| # | configuration | tok/s |\n|---|---|---|\n| 1 | fp16 eager (baseline) | 8.3 |\n| 2 | + CUDA graphs + skinny GEMV | 24.4 |\n| 3 | + AWQ 4-bit quantization | 29.9 |\n| 4 | + native HIP W4 GEMV kernel | 35.9 |\n| 5 | + M=1 GEMV for lm_head | 58.1 |\n| 6 | + direct-store kernel path | 59.1 |\n| 7 | + weight-cast caching | 62.5 |\n\n**TheRock** builds ROCm (HIP runtime, rocBLAS, Tensile) as native Windows binaries — this is what makes ROCm exist on Windows at all.`HSA_OVERRIDE_GFX_VERSION=10.3.0`\n\npresents any RDNA2 GPU as gfx1030; the HIP kernel ships as a**fat binary (gfx1030 + gfx1031 + gfx1032)** so the whole RX 6000 series runs native code.- PyTorch 2.12\n`+rocm7.15`\n\nlinks against that runtime →`torch.cuda.is_available() == True`\n\non RDNA2 Windows. - vLLM plugin\n`vllm_windows_rocm`\n\nregisters the tuned kernels: native HIP W4 GEMV for quantized linears, M=1 skinny GEMV for dense ones (including the huge tied lm_head), CUDA-graph safe (registered as real torch ops). `INSTALL.bat`\n\n(engine:`INSTALL.ps1`\n\n, manifest:`MANIFEST.json`\n\n) downloads the 4 prebuilt archives from GitHub Releases and the model from HuggingFace, installs base Python, fixes the venv, verifies with a benchmark. Idempotent: it only downloads what is missing.\n\n```\nC:\\Python311                                 Python 3.11.9\nC:\\TheRock\\.venv                             torch 2.12+rocm7.15 venv (vLLM 0.19.1)\nC:\\TheRock\\build\\dist\\rocm                   ROCm runtime libraries\nC:\\TheRock\\ROCM_VLLM_RUNTIME                 vLLM + plugin + rocBLAS + rocblas-bench\nC:\\vw_cext_build, C:\\vw_hipgemv_build        native HIP kernels\n%USERPROFILE%\\.cache\\huggingface             model weights\n```\n\nRelease archives (this repo's **Releases** tab, tag `v2.0`\n\n):\n\n| Archive | Size | Content |\n|---|---|---|\n`the-rock-venv.tar.zst` |\n1.34 GB | torch ROCm venv |\n`therock-rocm-dist.tar.zst` |\n0.85 GB | ROCm runtime |\n`vllm-stack.tar.zst` |\n0.14 GB | vLLM + plugin + rocBLAS + rocblas-bench.exe |\n`native-kernels.tar.zst` |\n~1 MB | HIP GEMV kernels (fat binary) |\n\n```\nvllm-rocm-windows-rdna2-oneclick/\n├── INSTALL.bat               one-click installer (entry point)\n├── INSTALL.ps1               installer engine (downloads, extracts, verifies)\n├── CHAT.bat                  opens the web chat (auto-starts the server)\n├── SERVE.bat                 starts the model server alone (OpenAI API)\n├── VERIFY.bat                all 3 verification checks in one run\n├── chat.html                 the browser chat UI (Thinking spinner + tok/s)\n├── MANIFEST.json             release archive names + extract targets\n├── plugin_overrides/         tuned plugin modules (awq_gemv, bf16_gemv)\n├── kernels/                  native HIP W4 GEMV source + prebuilt fat binary\n├── scripts/                  model server entry + benchmark + rebuild helper\n├── assets/                   verification screenshots\n└── results/                  raw benchmark logs + progression chart\n```\n\nEdit `config.bat`\n\n(written by the installer): set `SERVED_MODEL`\n\nto the model\nfolder and `MODEL_NAME`\n\nto the name shown in the chat / used by the API, then\nrun `CHAT.bat`\n\nagain. The benchmark uses `BENCH_MODEL`\n\n(same file):\n\n``` python\nC:\\TheRock\\.venv\\Scripts\\python.exe -c \"from huggingface_hub import snapshot_download; print(snapshot_download('<owner>/<repo>'))\"\n```\n\nprints the snapshot folder to put in `config.bat`\n\n.\n\n| Symptom | Fix |\n|---|---|\n`No module named 'vllm._C'` warnings |\nexpected — the Windows plugin loads native kernels instead |\n| Python installer exit 1601 | automatic NuGet fallback kicks in; nothing to do |\n| extraction \"Can't unlink\" errors | close stray Python processes, re-run `INSTALL.bat` (it resumes) |\n| low tok/s | close other GPU workloads; verify `overall_tok_s` ≥ 55 on a cold GPU |\n| non-RDNA2 GPU | installer warns; unvalidated outside RDNA2 |\n`tar` says \"Unrecognized archive format\" on the `.zst` files |\nold Windows 10 without zstd — install Windows updates (bsdtar 3.5+ bundles libzstd; Windows 11 is fine) |\n| chat page says \"Server error\" | the model is still loading — wait for \"Application startup complete\" in the SERVE window (~1 minute the first time) |\n| custom debugging | `set BENCH_MODE=eager` , `set VLLM_WIN_HIPGEMV=0` , `set VLLM_WIN_BF16_GEMV=0` |\n\n`scripts\\rebuild_kernel.py`\n\nrecompiles `kernels/src/gemv_w4.cu`\n\n(needs HIP SDK\n\n- MSVC). Only for non-RDNA2 targets or exotic torch ABIs; the shipped fat binary covers all RDNA2.\n\nBuilt on ROCm/TheRock, PyTorch ROCm and the vLLM project. Not affiliated with AMD.", "url": "https://wpnews.pro/news/native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16", "canonical_source": "https://github.com/sebastianmechno-sys/vllm-rocm-windows-rdna2", "published_at": "2026-08-18 10:11:51+00:00", "updated_at": "2026-08-18 10:41:10.545150+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["AMD", "Radeon RX 6750 XT", "ROCm 7.15", "vLLM", "Qwen3.5-4B", "sebastianmechno-sys", "PyTorch 2.12"], "alternates": {"html": "https://wpnews.pro/news/native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16", "markdown": "https://wpnews.pro/news/native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16.md", "text": "https://wpnews.pro/news/native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16.txt", "jsonld": "https://wpnews.pro/news/native-vllm-and-rocm-7-15-for-rx-6000-rdna2-on-windows-11-26-tflops-fp16.jsonld"}}