12:14
2026-09-11
gargighosh1.substack.com
artificial-intelligence
Evolution of Attention over the Years
A technical survey traces five rewrites of transformer attention since 2017, framing every variant as either storing less or reading less. It details how multi-query and grouped-query attention shrink…