{"slug": "advancing-quantization-for-r-r-models-with-microscaling-formats", "title": "Advancing Quantization for R&R Models with Microscaling Formats", "summary": "A new arXiv paper (2611.17501v1) reports the first significant speedups from native MXFP6 GEMMs, which quantize both weights and activations and compute directly on 6-bit elements, for industrial-scale Recommendation and Ranking (R&R) inference on AMD's MI350X architecture without breaching the 0.01% Normalized Entropy (NE) quality margin production ads models require. On two production ads ranking models, the implementation measured GEMM kernel speedups over FP16 of 1.33–1.55x for MXFP8 and 1.73–1.99x for MXFP6 across all supported linear layers, up to 2.90x for MXFP6 on individual layers, and end-to-end throughput gains from selective MXFP6 of 1.5–5.3% for Model A and 7.6–14.3% for Model B over their FP8 configurations. The authors also show MXFP8 matches row-wise FP8 accuracy nearly exactly, allowing the faster kernel to be chosen per GEMM shape.", "body_md": "arXiv:2611.17501v1 Announce Type: new \nAbstract: Recommendation and Ranking (R&R) models, particularly in digital advertising, require massive inference throughput but exhibit extreme sensitivity to precision loss. Unlike Large Language Models (LLMs) that frequently tolerate quantization-induced accuracy drops of 1% or more, production-grade R&R models enforce a stringent launch bar: model quality must remain within a 0.01% Normalized Entropy (NE) margin relative to the baseline, which commonly uses FP16 or FP32. This tight constraint has historically hindered the adoption of aggressive quantization in this domain. In this paper, we present an empirical evaluation of both MXFP8 and the 6-bit MXFP6 format for industrial-scale R&R inference on the AMD MI350X architecture.\n  To the best of our knowledge, this is the first paper, across R&R and LLM domains, that reports significant speedups from native MXFP6 GEMMs, which quantize both weights and activations and compute directly on 6-bit elements, without sacrificing model quality. We demonstrate that we can selectively apply this 6-bit format to maximize overall QPS gain without breaching the strict NE parity threshold. We further show that MXFP8 is a viable alternative to row-wise FP8: the two formats have nearly the same accuracy on our models, so the faster kernel can be chosen for each GEMM shape. On two production ads ranking models, our implementation shows measured GEMM kernel speedups over FP16 of 1.33--1.55x for MXFP8 and 1.73--1.99x for MXFP6 across all supported linear layers, and up to 2.90x for MXFP6 on individual layers, alongside end-to-end (E2E) inference throughput gains from selective MXFP6 of 1.5--5.3% for Model A and 7.6--14.3% for Model B over their FP8 configurations.", "url": "https://wpnews.pro/news/advancing-quantization-for-r-r-models-with-microscaling-formats", "canonical_source": "https://arxiv.org/abs/2611.17501", "published_at": "2026-11-16 05:00:00+00:00", "updated_at": "2026-11-16 05:19:42.466494+00:00", "lang": "en", "topics": ["machine-learning", "ai-infrastructure", "ai-chips", "ai-research"], "entities": ["AMD", "AMD MI350X", "MXFP6", "MXFP8", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/advancing-quantization-for-r-r-models-with-microscaling-formats", "markdown": "https://wpnews.pro/news/advancing-quantization-for-r-r-models-with-microscaling-formats.md", "text": "https://wpnews.pro/news/advancing-quantization-for-r-r-models-with-microscaling-formats.txt", "jsonld": "https://wpnews.pro/news/advancing-quantization-for-r-r-models-with-microscaling-formats.jsonld"}}