# FlashInfer on ROCm 10.1: The FlashInfer API with Automatic Kernel Routing

> Source: <https://rocm.blogs.amd.com/artificial-intelligence/flashinfer-release3/README.html>
> Published: 2026-11-20 00:00:00+00:00

**Key takeaway:** FlashInfer on ROCm 10.1 adds FP8 prefill, POD-Attention, fused MoE and AITER decode under CUDA graph capture, plus cascade attention, MLA and standalone RoPE, on gfx942 (MI300X/MI325X) and gfx950 (MI350X/MI355X) GPUs.

FlashInfer<sup>[1](#references)</sup> is an open-source library of attention and large language model (LLM) serving kernels. FlashInfer on ROCm™ brings its API to AMD Instinct™ GPUs, and this release continues from the prefill-attention release. Cascade attention, multi-head latent attention (MLA), and the standalone rotary position embedding (RoPE) ops, all in progress then, have now landed. New in this release are 8-bit floating-point (FP8) prefill, POD-Attention (prefill and decode overlapped in one kernel), fused mixture-of-experts (MoE), and AI Tensor Engine for ROCm (AITER) decode under CUDA graph capture. All of it runs on CDNA3 (gfx942: MI300X and MI325X)<sup>[2](#references)</sup> and CDNA4 (gfx950: MI350X and MI355X)<sup>[3](#references)</sup> GPUs.
