Grouped Value Attention

Researchers introduced Grouped Value Attention, a method that reduces the Transformer KV cache by reconstructing keys on demand rather than storing them at every decoding step. The technique builds on…