18:12
2026-08-25
dev.to
machine-learning
Did FP8 make the model dumber? A per-prompt regression check for quantized serving
An engineer at Jahn AI ran a per-prompt regression check before recommending FP8 quantization for Qwen3-8B serving on an RTX PRO 6000 Blackwell, which delivered a 1.5x throughput gain (1,725 to 2,597 …