AMD reported that UltraQuant, its 4-bit KV-cache method for the Qwen3.8-2.4T MXFP4 model on AMD Instinct MI355X GPUs, serves 29% more tokens per second at 24% lower per-token latency than the fastest …
AMD detailed model weight profiles that break down how many parameters go to embeddings, attention, and dense layers, explaining why the amd/Llama-3.1-8B-Instruct-FP8-KV checkpoint is 9.08 GB rather t…
AMD detailed a local quantization workflow using AMD Quark 0.12.post1 that compressed the Qwen3.6-35B-A3B Mixture-of-Experts model from BF16 to W4A16 on a single Strix Halo system, cutting model weigh…
AMD Quark now supports SVDQuant and native Hugging Face Diffusers integration for diffusion models, enabling 4-bit quantization of both weights and activations. On an AMD Instinct MI350 GPU, plain nat…
AMD Quark, a quantization library optimized for AMD Instinct MI350 GPUs, enables MXFP4 quantization for Diffusers and xDiT FLUX.1-dev image generation, achieving up to 1.92× speedup over BF16 eager an…
AMD announced the acceleration of large-scale LLM inference on its Instinct MI350X/MI355X GPUs using Eagle3 speculative decoding and AMD Quark quantization. The work enables lossless inference speedup…