Qwen 3.8 27B on RTX 5090 at 90-120tps

A developer published a llama-server configuration that runs a Qwen 3.8 27B NVFP4 model with MTP speculative decoding on an RTX 5090, reporting throughput of 90-120 tokens per second. The setup uses a 262,144-token context with q4_0 KV cache quantization, flash attention, and a draft-mtp speculative type capped at four draft tokens.

| | llama-server \ | | | --alias qwen3.8-27b-nvfp4-mtp-q8attn \ | | | -m "$HOME/.lmstudio/models/utautako/Qwen3.8-27B-NVFP4-MTP-Q8attn-GGUF/Qwen3.8-27B-NVFP4-MTP-Q8attn.gguf" \ | | | --mmproj "$HOME/.lmstudio/models/utautako/Qwen3.8-27B-NVFP4-MTP-Q8attn-GGUF/mmproj-Qwen3.8-27B-NVFP4-BF16.gguf" \ | | | --spec-type draft-mtp \ | | | --spec-draft-n-max 4 \ | | | -ngl 99 \ | | | -c 262144 \ | | | -ctk q4 0 \ | | | -ctv q4 0 \ | | | -b 2048 \ | | | -ub 512 \ | | | --host 127.0.0.1 \ | | | --port 1234 \ | | | --jinja \ | | | --reasoning-effort low \ | | | --kv-unified \ | | | -t 12 \ | | | -np 1 \ | | | --flash-attn on \ | | | --no-mmap |