I got Qwen3.8-27B running on dual RTX 3090s (no NVLink) under WSL2 — every pitfall I hit A developer detailed the process of running Qwen3.8-27B on dual RTX 3090s without NVLink under WSL2, achieving 170-210 tok/s on code/JSON. The setup required specific CUDA 13.0 toolchain, SGLang 0.5.17, and DSpark speculative decoding, overcoming pitfalls like vLLM incompatibility and WSL-specific crashes. Note: English translation assisted by an LLM. I'm not a native English speaker; all technical content, commands, and measurements come from my own setup. I spent a full week getting Qwen3.8-27B hybrid GDN architecture, 48 linear-attention + 16 full-attention layers, built-in MTP head running on my dual RTX 3090 24GB × 2 box — no NVLink, PCIe Gen4, Windows 10 + WSL2 Ubuntu-24.04 . I hit basically every trap in the book. This is the complete record: what broke, why, and the config that finally gave me 170-210 tok/s on code/JSON. | Item | Configuration | |---|---| | GPU | 2× NVIDIA RTX 3090 24GB GDDR6X, SM86/Ampere | | GPU interconnect | No NVLink PCIe Gen4 x8 bridge | | CPU / RAM | Ryzen 9 5950X / 64GB DDR4 | | OS | Windows 10 + WSL2 Ubuntu-24.04 | | Inference engine | SGLang 0.5.17 | | CUDA toolchain | CUDA 13.0 this exact version matters, see Pitfall 2 | | Model | Qwen3.8-27B-AWQ-INT4 cyankiwi quant, ~21GB | | Speculative decoding | DSpark 1.4B draft model | | Context length | 245,760 tokens | vLLM was the obvious first try. It did not go well: libnvrtc.so.13 nvidia-cuda-nvrtc was a 0.0.0a0 placeholder package . After force-reinstalling, flashinfer's sampling JIT demands the CUDA-13-only --host-stub-linkage-explicit while local nvcc is 12.8. Engine init crashes every time. Verdict: skip vLLM 0.27+ for Qwen3.8 on 3090s. The old Docker 0.25.1 works but gives no speedup. Switching to SGLang was worse: the model froze during weight loading — process hung, port never listened. I suspected parameters and tried dozens of combinations. Useless. Comparing keys in model.safetensors.index.json finally revealed the root cause: cyankiwi's Qwen3.8-AWQ-INT4 is a "half-quantized" checkpoint— the MTP head and some linear attn layers use plain weight instead of weight packed , unlike Qwen3.6's fully-quantized checkpoints. Same architecture config: Qwen3.6 runs, Qwen3.8 dies on load. The real fix was toolchain matching: | CUDA | Result | |---|---| | 12.8 | weight loading hangs | | 13.3 | NV ATOMIC RELAXED macro removed → verify-graph compilation fails | 13.0 ✅ | perfect match for torch cu130; keeps the cccl macros flashinfer 0.6.15 needs | Plus the no-NVLink triple fix: --disable-custom-all-reduce avoid "peer access is not supported" --mm-feature-transport cpu bypass pidfd getfd crash in CUDA IPC under WSL NCCL P2P DISABLE=1 must disable P2P without NVLink And a SymmMem patch: set MultimemAllGatherer.enabled=False in logits processor.py to bypass the SIGFPE from torch.distributed. symmetric memory.rendezvous on 3090s. It ran At ~10 tok/s . Unusable. | Approach | Acceptance | Speed | Verdict | |---|---|---|---| | No speculation baseline | — | ~63 tok/s | start | | MTP/EAGLE built-in head | 1.07-1.27 | ~37 tok/s | ❌ negative — INT4 quantization damaged the BF16 head | | NGRAM | — | ~56 tok/s | ❌ +50% on JSON only, -30% on normal text | | ReplaySSM | 1.45→1.55 | 40→43 tok/s | ❌ negligible | DSpark 1.4B draft model | 3.7-4.2 | 170-210 tok/s | ✅✅✅ | DSpark was the only thing that worked. An independent 1.4B draft model proposes a 7-token candidate block per step; the main model verifies the block in one pass. Acceptance of 3.7-4.2 means ~4 tokens accepted per verification — 3x better than MTP on this quantized model. Symptom : service starts, but server args= log shows default values for everything — mm feature transport='cuda ipc' , speculative algorithm=None . Cause : a trailing \ went missing in the launch script; bash truncates the command there and silently discards every parameter after it . The service boots with bare defaults, which crashes on WSL without NVLink. Lesson : after editing the launch script, verify with grep server args= that your parameters actually took effect before tuning anything. This may be the truth behind 80% of "my parameters don't work" reports. Qwen3.8's GDN kernels need CUDA-13 compilation. 12.8 hangs on load, 13.3 fails to compile, 13.0 is exactly right. Check which cu version your SGLang torch was built against cu130 , then match the toolchain exactly. --enable-torch-compile crashes on GDN launcher missing ' grid 2' — inductor can't compile causal conv1d .Qwen3.8's built-in MTP head is BF16, but this INT4 quant quantized the head too — it misjudges on INT4 hidden states, acceptance never rises 1.07-1.27 . Not a parameter problem; a quantization problem. Either use a W4A16 model that keeps the BF16 head limited gain without P2P or go DSpark. --mamba-radix-cache-strategy extra buffer recommended to avoid GDN degradation : --chunked-prefill-size 2048 recommended for concurrency : for single-request + 50-90K token agent contexts it quadruples prefill iterations and slows TTFT. Test your own workload before copying official parameters. --mem-fraction-static too high + large context → no headroom for CUDA graph capture → verify-graph hangs log stalls at Capturing batches 0% , GPU 100% but CPU time frozen . Fix: 0.8 DSpark also needs VRAM for the 1.4B draft , or --cuda-graph-backend-prefill disabled . How to tell a hang from progress : ps -o pid,pcpu,time — CPU TIME growing = healthy, frozen = deadlock. Don't kill the process prematurely; graph capture legitimately takes 90-120s. --served-model-name must be a short name Clients send short model names; if the server only exposes the full path → detokenizer hangs , health checks fail, everything times out. Add --served-model-name