00:00
2026-06-15
martinalderson.com
large-language-models
A brief history of KV cache compression developments
The memory needed to store one token of context in large language models has fallen by roughly 100x since 2017, driven by advances in KV cache compression techniques such as Multi-Query Attention (MQA…