{"slug": "kv-cache-compression-through-the-lens-of-transform-coding", "title": "KV Cache Compression Through the Lens of Transform Coding", "summary": "Researchers introduced Attention-Aware Transform Coding (AATC), a KV cache compression method that reduces memory use by approximately 5.8x while maintaining near-lossless accuracy on models like Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, outperforming baseline quantization methods across benchmarks including LongBench, RULER, GSM8K, MMLU-Pro, and MATH-500.", "body_md": "arXiv:2608.14191v1 Announce Type: new\nAbstract: The key-value (KV) cache stores information from past tokens and is a major memory bottleneck in long-context inference. Existing quantization methods address this bottleneck by representing the KV cache uniformly with lower-precision data types and designing quantization schemes to minimize reconstruction error in the cache itself, without accounting for how that error propagates through attention mechanisms. We prove that, under a white-noise quantization model, the expected attention-aware distortion decomposes into additive key and value contributions that factor across tokens and channels. Building on transform coding and reverse water-filling, which are classical tools from signal processing and rate-distortion theory, we introduce Attention-Aware Transform Coding (AATC), which allocates bits over a calibration set to minimize attention-aware distortion. On Llama-3.1-8B-Instruct and Qwen-2.5-7B-Instruct, evaluated across LongBench, RULER, GSM8K, MMLU-Pro, and MATH-500, our method achieves near-lossless accuracy at approximately $5.8\\times$ compression, whereas each baseline degrades in at least some settings.", "url": "https://wpnews.pro/news/kv-cache-compression-through-the-lens-of-transform-coding", "canonical_source": "https://www.machinebrief.com/news/kv-cache-compression-through-the-lens-of-transform-coding-hss5", "published_at": "2026-08-17 04:00:00+00:00", "updated_at": "2026-08-17 04:42:30.471865+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-research"], "entities": ["Llama-3.1-8B-Instruct", "Qwen-2.5-7B-Instruct", "LongBench", "RULER", "GSM8K", "MMLU-Pro", "MATH-500"], "alternates": {"html": "https://wpnews.pro/news/kv-cache-compression-through-the-lens-of-transform-coding", "markdown": "https://wpnews.pro/news/kv-cache-compression-through-the-lens-of-transform-coding.md", "text": "https://wpnews.pro/news/kv-cache-compression-through-the-lens-of-transform-coding.txt", "jsonld": "https://wpnews.pro/news/kv-cache-compression-through-the-lens-of-transform-coding.jsonld"}}