HP Z8 Fury G6i -- Perfect Qwen Flash Next setup (2x RTX Pro 6000s)

HP's Z8 Fury G6i workstation, equipped with dual RTX PRO 6000 Blackwell GPUs (96 GB each), 125 GB RAM, and a 48-core Intel Xeon X658X, serves Qwen3.8-Flash-Next-FP8 (125B MoE) at up to 218 tokens/s single-stream and 345 tokens/s at concurrency 8 with 1M context and MTP enabled, according to a technical setup guide. The configuration uses vLLM engine version 0.1.dev20073, offloads the 51B n-gram table to host RAM, and achieves 98-100% MTP acceptance with a mean acceptance length of ~4.0.

Be sure to check out our full review and other content featuring the HP Z8 Fury G6i Host: 2x RTX PRO 6000 Blackwell 96 GB , 125 GB RAM, 48 cores Intel Xeon X658X Model: Qwen/Qwen3.8-Flash-Next-FP8 125B MoE / 6B active, 51B n-gram table, 4B MTP Engine: vLLM vllm/vllm-openai:qwen38-flash-next 0.1.dev20073 Status: SERVING on port 8000, model qwen3.8-flash-next , 1M context, MTP enabled | Component | Choice | Why | |---|---|---| | Checkpoint | FP8 187 GB | BF16 is 335 GB won’t fit 2x96GB | | n-gram table | VLLM PLE CPU OFFLOAD=1 | 51B params moved to host RAM; GPU holds only ~67 GiB weights | | MTP | num speculative tokens: 3 | 2x decode speedup; acceptance 98-100% | | Context | 1M via YaRN | Fits at 0.95 gpu-mem-util 95.5 GiB/GPU | | TP | 2 | 2 GPUs | YaRN needs more testing, really, probably. | Config | Single-stream | Concurrency 8 | TTFT | |---|---|---|---| | 262K, no MTP | 107 tok/s | 238 tok/s | 76 ms | | 262K, MTP-3 | 218 tok/s | 265 tok/s | ~100 ms | 1M, MTP-3 | 218 tok/s | 345 tok/s | ~100 ms | MTP acceptance: 98-100%, mean acceptance length ~4.0 near-perfect drafting . Here’s my startup script I’m a little surprised this was so easy? and reachable on this system? bash /bin/bash Launch Qwen3.8-Flash-Next-FP8 on bighp 2x RTX PRO 6000 Blackwell 96GB Usage: ./launch qwen38.sh baseline|1m mtp|nomtp set -euo pipefail MODE="${1:-baseline}" baseline 262K or 1m MTP="${2:-mtp}" mtp or nomtp NAME="vllm-qwen38-fpn" MODEL DIR="/home/w/models/Qwen3.8-Flash-Next-FP8" PORT=8000 Common args ARGS= --model /models/Qwen3.8-Flash-Next-FP8 --tensor-parallel-size 2 --gpu-memory-utilization 0.90 --max-num-seqs 256 --enable-prefix-caching --no-enable-flashinfer-autotune --enable-auto-tool-choice --tool-call-parser qwen3 xml --reasoning-parser qwen3 --served-model-name qwen3.8-flash-next --port "$PORT" if "$MODE" = "1m" ; then ARGS+= --max-model-len 1000000 --gpu-memory-utilization 0.95 --hf-overrides '{"text config": {"rope parameters": {"mrope interleaved": true, "mrope section": 11, 11, 10 , "rope type": "yarn", "rope theta": 10000000, "partial rotary factor": 0.25, "factor": 4.0, "original max position embeddings": 262144}}}' else ARGS+= --max-model-len 262144 fi if "$MTP" = "mtp" ; then ARGS+= --speculative-config '{"method":"mtp","num speculative tokens":3}' fi echo "=== Launching vLLM: mode=$MODE mtp=$MTP ===" echo "docker run -d --name $NAME ... ${ARGS }" Remove old container if present docker rm -f "$NAME" /dev/null 2 &1 || true docker run -d --name "$NAME" --restart unless-stopped \ --ipc=host --gpus all \ --shm-size 16g \ -e VLLM PLE CPU OFFLOAD=1 \ -e VLLM ALLOW LONG MAX MODEL LEN=1 \ -v "$MODEL DIR":/models/Qwen3.8-Flash-Next-FP8:ro \ -p "$PORT":8000 \ vllm/vllm-openai:qwen38-flash-next \ "${ARGS @ }" echo "Container $NAME started. Follow logs: docker logs -f $NAME"