01:03
2026-08-16
changyi.fun
machine-learning
Attention Through Arithmetic Intensity
A technical analysis by an unnamed author, inspired by a post from Su Jianlin, derives the arithmetic intensity (FLOPs per byte) of attention during single-token decode, finding MHA has an AI of exact…