{"slug": "hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s", "title": "HP Z8 Fury G6i -- Perfect Qwen Flash Next setup (2x RTX Pro 6000s)", "summary": "HP's Z8 Fury G6i workstation, equipped with dual RTX PRO 6000 Blackwell GPUs (96 GB each), 125 GB RAM, and a 48-core Intel Xeon X658X, serves Qwen3.8-Flash-Next-FP8 (125B MoE) at up to 218 tokens/s single-stream and 345 tokens/s at concurrency 8 with 1M context and MTP enabled, according to a technical setup guide. The configuration uses vLLM engine version 0.1.dev20073, offloads the 51B n-gram table to host RAM, and achieves 98-100% MTP acceptance with a mean acceptance length of ~4.0.", "body_md": "*Be sure to check out our full review and other content featuring the HP Z8 Fury G6i*\n\n**Host:** 2x RTX PRO 6000 Blackwell (96 GB), 125 GB RAM, 48 cores Intel Xeon X658X\n\n**Model:** Qwen/Qwen3.8-Flash-Next-FP8 (125B MoE / 6B active, 51B n-gram table, 4B MTP)\n\n**Engine:** vLLM `vllm/vllm-openai:qwen38-flash-next`\n\n(0.1.dev20073)\n\n**Status:** SERVING on port 8000, model `qwen3.8-flash-next`\n\n, 1M context, MTP enabled\n\n| Component | Choice | Why |\n|---|---|---|\n| Checkpoint | FP8 (187 GB) | BF16 is 335 GB won’t fit 2x96GB |\n| n-gram table | `VLLM_PLE_CPU_OFFLOAD=1` |\n51B params moved to host RAM; GPU holds only ~67 GiB weights |\n| MTP | `num_speculative_tokens: 3` |\n2x decode speedup; acceptance 98-100% |\n| Context | 1M via YaRN | Fits at 0.95 gpu-mem-util (95.5 GiB/GPU)* |\n| TP | 2 | 2 GPUs |\n\n*YaRN needs more testing, really, probably.\n\n| Config | Single-stream | Concurrency 8 | TTFT |\n|---|---|---|---|\n| 262K, no MTP | 107 tok/s | 238 tok/s | 76 ms |\n| 262K, MTP-3 | 218 tok/s | 265 tok/s | ~100 ms |\n1M, MTP-3 |\n218 tok/s |\n345 tok/s |\n~100 ms |\n\nMTP acceptance: 98-100%, mean acceptance length ~4.0 (near-perfect drafting).\n\nHere’s my startup script (I’m a little surprised this was so easy? and reachable on this system?)\n\n``` bash\n#!/bin/bash\n# Launch Qwen3.8-Flash-Next-FP8 on bighp (2x RTX PRO 6000 Blackwell 96GB)\n# Usage: ./launch_qwen38.sh [baseline|1m] [mtp|nomtp]\nset -euo pipefail\n\nMODE=\"${1:-baseline}\"   # baseline (262K) or 1m\nMTP=\"${2:-mtp}\"         # mtp or nomtp\n\nNAME=\"vllm-qwen38-fpn\"\nMODEL_DIR=\"/home/w/models/Qwen3.8-Flash-Next-FP8\"\nPORT=8000\n\n# Common args\nARGS=(\n  --model /models/Qwen3.8-Flash-Next-FP8\n  --tensor-parallel-size 2\n  --gpu-memory-utilization 0.90\n  --max-num-seqs 256\n  --enable-prefix-caching\n  --no-enable-flashinfer-autotune\n  --enable-auto-tool-choice\n  --tool-call-parser qwen3_xml\n  --reasoning-parser qwen3\n  --served-model-name qwen3.8-flash-next\n  --port \"$PORT\"\n)\n\nif [ \"$MODE\" = \"1m\" ]; then\n  ARGS+=(\n    --max-model-len 1000000\n    --gpu-memory-utilization 0.95\n    --hf-overrides '{\"text_config\": {\"rope_parameters\": {\"mrope_interleaved\": true, \"mrope_section\": [11, 11, 10], \"rope_type\": \"yarn\", \"rope_theta\": 10000000, \"partial_rotary_factor\": 0.25, \"factor\": 4.0, \"original_max_position_embeddings\": 262144}}}'\n  )\nelse\n  ARGS+=(--max-model-len 262144)\nfi\n\nif [ \"$MTP\" = \"mtp\" ]; then\n  ARGS+=(\n    --speculative-config '{\"method\":\"mtp\",\"num_speculative_tokens\":3}'\n  )\nfi\n\necho \"=== Launching vLLM: mode=$MODE mtp=$MTP ===\"\necho \"docker run -d --name $NAME ... ${ARGS[*]}\"\n\n# Remove old container if present\ndocker rm -f \"$NAME\" >/dev/null 2>&1 || true\n\ndocker run -d --name \"$NAME\" --restart unless-stopped \\\n  --ipc=host --gpus all \\\n  --shm-size 16g \\\n  -e VLLM_PLE_CPU_OFFLOAD=1 \\\n  -e VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \\\n  -v \"$MODEL_DIR\":/models/Qwen3.8-Flash-Next-FP8:ro \\\n  -p \"$PORT\":8000 \\\n  vllm/vllm-openai:qwen38-flash-next \\\n  \"${ARGS[@]}\"\n\necho \"Container $NAME started. Follow logs: docker logs -f $NAME\"\n```\n\n", "url": "https://wpnews.pro/news/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s", "canonical_source": "https://forum.level1techs.com/t/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s/254584#post_1", "published_at": "2026-08-28 20:50:13+00:00", "updated_at": "2026-08-28 21:20:46.437725+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["HP Z8 Fury G6i", "NVIDIA RTX PRO 6000 Blackwell", "Intel Xeon X658X", "Qwen3.8-Flash-Next-FP8", "vLLM"], "alternates": {"html": "https://wpnews.pro/news/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s", "markdown": "https://wpnews.pro/news/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s.md", "text": "https://wpnews.pro/news/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s.txt", "jsonld": "https://wpnews.pro/news/hp-z8-fury-g6i-perfect-qwen-flash-next-setup-2x-rtx-pro-6000s.jsonld"}}