{"slug": "flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing", "title": "FlashInfer on ROCm 10.1: The FlashInfer API with Automatic Kernel Routing", "summary": "FlashInfer on ROCm 10.1 brings the FlashInfer attention and LLM serving kernel API to AMD Instinct GPUs with automatic kernel routing, adding FP8 prefill, POD-Attention (prefill and decode overlapped in one kernel), fused mixture-of-experts (MoE), and AITER decode under CUDA graph capture. The release also lands cascade attention, multi-head latent attention (MLA), and standalone rotary position embedding (RoPE) ops that were in progress in the earlier prefill-attention release. It runs on CDNA3 (gfx942: MI300X and MI325X) and CDNA4 (gfx950: MI350X and MI355X) GPUs.", "body_md": "FlashInfer<sup>[1](#references)</sup> is an open-source library of attention and large language model (LLM) serving kernels. FlashInfer on ROCm™ brings its API to AMD Instinct™ GPUs, and this release continues from the prefill-attention release. Cascade attention, multi-head latent attention (MLA), and the standalone rotary position embedding (RoPE) ops, all in progress then, have now landed. New in this release are 8-bit floating-point (FP8) prefill, POD-Attention (prefill and decode overlapped in one kernel), fused mixture-of-experts (MoE), and AI Tensor Engine for ROCm (AITER) decode under CUDA graph capture. All of it runs on CDNA3 (gfx942: MI300X and MI325X)<sup>[2](#references)</sup> and CDNA4 (gfx950: MI350X and MI355X)<sup>[3](#references)</sup> GPUs.", "url": "https://wpnews.pro/news/flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing", "canonical_source": "https://rocm.blogs.amd.com/artificial-intelligence/flashinfer-release3/README.html", "published_at": "2026-11-20 00:00:00+00:00", "updated_at": "2026-11-20 15:43:52.987469+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-chips", "large-language-models", "machine-learning", "ai-tools"], "entities": ["FlashInfer", "ROCm", "AMD Instinct", "MI300X", "MI325X", "MI350X", "MI355X", "AITER"], "also_reported_by": [], "takeaway": "FlashInfer on ROCm 10.1 adds FP8 prefill, POD-Attention, fused MoE and AITER decode under CUDA graph capture, plus cascade attention, MLA and standalone RoPE, on gfx942 (MI300X/MI325X) and gfx950 (MI350X/MI355X) GPUs.", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing", "markdown": "https://wpnews.pro/news/flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing.md", "text": "https://wpnews.pro/news/flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing.txt", "jsonld": "https://wpnews.pro/news/flashinfer-on-rocm-10-1-the-flashinfer-api-with-automatic-kernel-routing.jsonld"}}