Qwen 3.8 Flash Next NVFP4 on a single RTX Pro 6000 and system memory

A user benchmarked the Qwen 3.8 Flash Next NVFP4 model on a single RTX Pro 6000 GPU, achieving 81-88 tokens per second in a single session and scaling to 341 tokens per second with six concurrent clients. The model requires 100GB of system memory for its 51B n-gram table, with prompt preprocessing compute-bound at 12,000 tokens per second even at 100k context.

Followed the documentation on primitive-ai/Qwen3.8-Flash-Next-NVFP4 · Hugging Face https://huggingface.co/primitive-ai/Qwen3.8-Flash-Next-NVFP4 Here is my full report Claude Artifact https://claude.ai/code/artifact/712952eb-f092-48fd-88fd-0762f1a3f837 . Basic 81- 88 tokens per second on a single session. Scales to a throughput of 341 tokens per second when running 6 concurrent client. Interesting piece is the prompt preprocessing is GPU compute bound at 12000 tokens per second even when context is scaled up to 100k. This model rules but you need to have 100GB spare system memory to hold the 51B n-gram table. 1 Like