A single AMD MI300X GPU with 192 GB of HBM3 now serves DeepSeek's 284B-parameter DeepSeek-V4-Flash-0731 checkpoint in mixed FP4+FP8 format, requiring nine patch overlays against a vLLM ROCm nightly pl…
A production configuration for running DeepSeek-V4-Flash-0731 on a single AMD MI300X GPU achieves 168.6 tok/s median single-stream decode and 542 tok/s aggregate across 8 concurrent streams, with the …
AMD shows how to serve the pre-quantized amd/Kimi-K2.5-MXFP4 checkpoint on AMD Instinct MI355X GPUs using ATOM, a lightweight vLLM-like framework that integrates AITER kernels and exposes an OpenAI-co…
AMD has optimized MiniMax-M3 inference on Instinct MI355X GPUs using ATOM, AITER, and ATOMesh, achieving improvements in serving throughput, token latency, and accuracy through online quantization, sp…
AMD Quark, a quantization library optimized for AMD Instinct MI350 GPUs, enables MXFP4 quantization for Diffusers and xDiT FLUX.1-dev image generation, achieving up to 1.92× speedup over BF16 eager an…
AMD announced a new kernel family, LDS-Pipelined Split-K GEMM, that accelerates LLM inference on AMD GPUs by optimizing decode-time GEMMs with small M and large N/K dimensions. The technique achieves …
MoonMath AI open-sourced a bf16 forward attention kernel for AMD MI300X GPUs that outperforms AMD's AITER v3 on every tested shape and rounding mode, achieving up to 1.26x speedup. The kernel, written…
A team of kernel engineers developed a bf16 forward attention kernel for AMD MI300X GPUs using HIP, outperforming AMD's own AITER v3 library by up to 1.26× across various token lengths and rounding mo…
AMD shipped ATOM + ATOMesh, a ROCm-native LLM serving stack for Instinct GPUs that implements prefill/decode disaggregation, splitting the two inference phases onto separate GPU pools to optimize for …