{"slug": "did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized", "title": "Did FP8 make the model dumber? A per-prompt regression check for quantized serving", "summary": "An engineer at Jahn AI ran a per-prompt regression check before recommending FP8 quantization for Qwen3-8B serving on an RTX PRO 6000 Blackwell, which delivered a 1.5x throughput gain (1,725 to 2,597 tok/s at concurrency 32 with vLLM). The check compares greedy-decoding outputs between BF16 and FP8 across 20 prompts, using exact equality and difflib similarity to flag material differences, and passed for the tested workload. The engineer notes the method validates a prompt profile, not the model generally, and that a kernel assertion in the default FP8 path on sm_120 required a workaround.", "body_md": "FP8 gave us a clean 1.5x on Qwen3-8B serving throughput on an RTX PRO 6000 Blackwell (1,725 to 2,597 tok/s at concurrency 32, vLLM). The uncomfortable question is always the same: did the model get dumber. This post is the exact check we ran before recommending the switch, with numbers, so you can run the same one.\n\nStandard benchmarks (MMLU and friends) are noisy instruments for quantization deltas at 8B scale. Score movement inside the error bars tells you nothing about whether YOUR prompts changed behavior. What you actually want to know is narrower: on the workload you serve, does the FP8 checkpoint produce materially different outputs than BF16, and are any of the differences wrong.\n\nThat is answerable directly, cheaply, and per prompt.\n\nBoth configurations run the same fixed workload: 20 prompts covering reasoning, code, summarization, translation, extraction, classification, math, and instruction following. Greedy decoding, temperature 0, 256-token cap, streamed. Greedy matters: it removes sampling noise, so any output difference is attributable to the numerics.\n\nThen a three-stage comparison:\n\n`outputs_bf16[i] == outputs_fp8[i]`\n\n. Anything identical is settled.`difflib.SequenceMatcher.ratio()`\n\nsorts near-identical wording drift from real divergence.The core loop is small:\n\n``` python\nimport difflib, json\n\nbf16 = json.load(open(\"vllm_bf16_conc1.texts.json\"))\nfp8  = json.load(open(\"vllm_fp8_conc1.texts.json\"))\n\nfor i, (a, b) in enumerate(zip(bf16, fp8)):\n    if a == b:\n        print(i, \"identical\")\n        continue\n    r = difflib.SequenceMatcher(None, a, b).ratio()\n    print(i, f\"similarity {r:.3f}\")\n    # non-identical pairs go to side-by-side review\n```\n\nThat last line is the acceptance bar. Minor stylistic wobble is expected from a numerics change; a flipped fact is a blocker. This profile passed, so the 1.5x was free for this workload.\n\nTwo honest caveats. First, this validates a prompt profile, not the model in general: different domains, longer contexts, or sampled decoding need their own pass. Second, greedy-decoding equality is a strict signal but not a complete one; if you serve with sampling, run the review stage on sampled pairs too and expect more (benign) divergence.\n\nGetting FP8 to run at all on workstation-class Blackwell (sm_120) required routing around a kernel assertion in the default FP8 path. If you are on the same silicon and vLLM refuses to load the FP8 checkpoint, that is a known class of problem rather than something wrong with your setup.\n\nThe full report this check belongs to, with the raw CSVs, environment manifest, and the one-command reproduction script, is here: [https://conatus.jahn.ai/ai-engineering/sample-report](https://conatus.jahn.ai/ai-engineering/sample-report)", "url": "https://wpnews.pro/news/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized", "canonical_source": "https://dev.to/conatusai/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized-serving-595f", "published_at": "2026-08-25 18:12:24+00:00", "updated_at": "2026-08-25 18:44:07.045578+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-infrastructure", "mlops", "developer-tools"], "entities": ["Jahn AI", "Qwen3-8B", "RTX PRO 6000 Blackwell", "vLLM", "FP8", "BF16", "difflib"], "alternates": {"html": "https://wpnews.pro/news/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized", "markdown": "https://wpnews.pro/news/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized.md", "text": "https://wpnews.pro/news/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized.txt", "jsonld": "https://wpnews.pro/news/did-fp8-make-the-model-dumber-a-per-prompt-regression-check-for-quantized.jsonld"}}