Hello-Fri-End (auto-discovered)

Tim Dettmers et al. in their 2022 LLM.int8() paper found that large outlier features emerge across all transformer layers as models scale, causing naive INT8 quantization to break down. They introduce…

Transformer models develop outlier channels—feature dimensions with unusually large values in weights and activations—due to the softmax normalization in attention layers, which forces tokens to assig…

Integer quantization reduces memory and energy consumption in large language models by representing weights and activations with fewer bits, enabling 70B models to fit on a single GPU in 4-bit precisi…