# HP Z8 Fury G6i -- Perfect Qwen Flash Next setup (2x RTX Pro 6000s)

> Source: <https://forum.level1techs.com/t/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s/254584#post_1>
> Published: 2026-08-28 20:50:13+00:00

*Be sure to check out our full review and other content featuring the HP Z8 Fury G6i*

**Host:** 2x RTX PRO 6000 Blackwell (96 GB), 125 GB RAM, 48 cores Intel Xeon X658X

**Model:** Qwen/Qwen3.8-Flash-Next-FP8 (125B MoE / 6B active, 51B n-gram table, 4B MTP)

**Engine:** vLLM `vllm/vllm-openai:qwen38-flash-next`

(0.1.dev20073)

**Status:** SERVING on port 8000, model `qwen3.8-flash-next`

, 1M context, MTP enabled

| Component | Choice | Why |
|---|---|---|
| Checkpoint | FP8 (187 GB) | BF16 is 335 GB won’t fit 2x96GB |
| n-gram table | `VLLM_PLE_CPU_OFFLOAD=1` |
51B params moved to host RAM; GPU holds only ~67 GiB weights |
| MTP | `num_speculative_tokens: 3` |
2x decode speedup; acceptance 98-100% |
| Context | 1M via YaRN | Fits at 0.95 gpu-mem-util (95.5 GiB/GPU)* |
| TP | 2 | 2 GPUs |

*YaRN needs more testing, really, probably.

| Config | Single-stream | Concurrency 8 | TTFT |
|---|---|---|---|
| 262K, no MTP | 107 tok/s | 238 tok/s | 76 ms |
| 262K, MTP-3 | 218 tok/s | 265 tok/s | ~100 ms |
1M, MTP-3 |
218 tok/s |
345 tok/s |
~100 ms |

MTP acceptance: 98-100%, mean acceptance length ~4.0 (near-perfect drafting).

Here’s my startup script (I’m a little surprised this was so easy? and reachable on this system?)

``` bash
#!/bin/bash
# Launch Qwen3.8-Flash-Next-FP8 on bighp (2x RTX PRO 6000 Blackwell 96GB)
# Usage: ./launch_qwen38.sh [baseline|1m] [mtp|nomtp]
set -euo pipefail

MODE="${1:-baseline}"   # baseline (262K) or 1m
MTP="${2:-mtp}"         # mtp or nomtp

NAME="vllm-qwen38-fpn"
MODEL_DIR="/home/w/models/Qwen3.8-Flash-Next-FP8"
PORT=8000

# Common args
ARGS=(
  --model /models/Qwen3.8-Flash-Next-FP8
  --tensor-parallel-size 2
  --gpu-memory-utilization 0.90
  --max-num-seqs 256
  --enable-prefix-caching
  --no-enable-flashinfer-autotune
  --enable-auto-tool-choice
  --tool-call-parser qwen3_xml
  --reasoning-parser qwen3
  --served-model-name qwen3.8-flash-next
  --port "$PORT"
)

if [ "$MODE" = "1m" ]; then
  ARGS+=(
    --max-model-len 1000000
    --gpu-memory-utilization 0.95
    --hf-overrides '{"text_config": {"rope_parameters": {"mrope_interleaved": true, "mrope_section": [11, 11, 10], "rope_type": "yarn", "rope_theta": 10000000, "partial_rotary_factor": 0.25, "factor": 4.0, "original_max_position_embeddings": 262144}}}'
  )
else
  ARGS+=(--max-model-len 262144)
fi

if [ "$MTP" = "mtp" ]; then
  ARGS+=(
    --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
  )
fi

echo "=== Launching vLLM: mode=$MODE mtp=$MTP ==="
echo "docker run -d --name $NAME ... ${ARGS[*]}"

# Remove old container if present
docker rm -f "$NAME" >/dev/null 2>&1 || true

docker run -d --name "$NAME" --restart unless-stopped \
  --ipc=host --gpus all \
  --shm-size 16g \
  -e VLLM_PLE_CPU_OFFLOAD=1 \
  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
  -v "$MODEL_DIR":/models/Qwen3.8-Flash-Next-FP8:ro \
  -p "$PORT":8000 \
  vllm/vllm-openai:qwen38-flash-next \
  "${ARGS[@]}"

echo "Container $NAME started. Follow logs: docker logs -f $NAME"
```