02:27
2026-09-27
dev.to
machine-learning
FlashAttention-2 from PyTorch to Triton
A developer implemented the FlashAttention-2 forward pass twice — first as a deliberately slow PyTorch reference with a Python double loop over tiles, then as a Triton kernel — to teach kernel optimiz…