Attention Variants and Memory Efficiency
A trained checkpoint's attention architecture — multi-head, grouped-query, multi-query, sliding-window, or multi-head latent attention — fixes its key-value cache memory requirements at inference, so serving teams must i…