GQA

An engineer provides a formula to estimate KV cache memory consumption for large language models, showing that the KV cache often becomes the bottleneck before model weights. For Llama 3.1 70B at 128K…

Researchers introduced a semi-supervised framework that trains a lightweight reasoning-correctness classifier on a few labeled examples to verify LLM intermediate reasoning traces, then uses entropy-b…

A systematic study of query, key, and value (QKV) projection variants in Transformers found that sharing the key and value projections (Q-K=V) performs on par with or better than the standard three-pr…