Making FlashAttention-4 faster for inference
Modal AI engineers Charles Frye and David Wang optimized FlashAttention-4 for large language model inference, focusing on decode-heavy workloads dominated by memory bandwidth-limited token generation. The team adjusted p…