{"slug": "gaac-lossless-kv-cache-compression-for-faster-llm-inference", "title": "GAAC: Lossless KV Cache Compression for Faster LLM Inference", "summary": "A paper submitted to arXiv on 28 Oct 2026 presents Group-Adaptive Additive Codec (GAAC), a lossless KV cache compression and serving system that achieves 1.405-1.407x BF16 KV compression across five tested models while preserving KV values bit-for-bit. On NVIDIA DGX Spark, GAAC speeds decode attention at a 64K-token context by 1.19-1.27x and end-to-end W4A16 decode steps by 1.04-1.15x, cuts resident KV memory by roughly 26% and raises aggregate throughput 13-29% under comparable KV-memory budgets; on Apple M4 it delivers 1.02-1.25x attention speedups. GAAC encodes BF16 exponents with calibrated per-channel base-plus-offset codes and uses a fused attention kernel that reconstructs KV tiles on the fly, avoiding full-cache decompression.", "body_md": "# Computer Science > Artificial Intelligence\n\n  [Submitted on 28 Oct 2026]\n\n# Title:GAAC: Lossless KV Compression for Bandwidth-Efficient LLM Inference on Personal Devices\n\n[View PDF](https://arxiv.org/pdf/2610.38210)\n\n[HTML (experimental)](https://arxiv.org/html/2610.38210v1)\n\nAbstract:Weight-only, lossy quantization substantially reduces the storage and memory requirements for serving large language models (LLMs), but the growing key-value (KV) cache remains a bottleneck for long-context inference on personal devices. We present Group-Adaptive Additive Codec (GAAC), a lossless KV compression and serving system that exploits exponent redundancy to reduce storage and accelerate attention without introducing KV approximation. GAAC encodes BF16 exponents using calibrated per-channel base-plus-offset codes over contiguous ranges. GAAC adapts code widths across small groups while preserving KV values bit-for-bit. A fused attention kernel reconstructs KV tiles on the fly, avoiding full-cache decompression. Block-wise compression and fixed-capacity slots support incremental cache updates and paged serving. Across five tested models, GAAC achieves 1.405-1.407$\\times$ lossless BF16 KV compression. We also study compression performance on low-precision tensors FP8 E4M3 caches ( 1.09-1.14$\\times$) and E5M2 caches (1.25-1.30$\\times$). On NVIDIA DGX Spark, GAAC accelerates decode attention at a 64K-token context by 1.19-1.27$\\times$ and end-to-end W4A16 decode steps by 1.04-1.15$\\times$. In serving, it reduces resident KV memory by approximately 26% and increases aggregate throughput by 13-29% under comparable KV-memory budgets. On Apple M4, GAAC achieves 1.02-1.25$\\times$ attention speedups, demonstrating portability beyond NVIDIA GPUs.\n    \n\n### Additional Features\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/gaac-lossless-kv-cache-compression-for-faster-llm-inference", "canonical_source": "https://arxiv.org/abs/2610.38210", "published_at": "2026-10-31 15:34:12+00:00", "updated_at": "2026-10-31 15:51:01.348722+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure", "machine-learning"], "entities": ["GAAC", "Group-Adaptive Additive Codec", "arXiv", "NVIDIA DGX Spark", "Apple M4", "BF16", "FP8 E4M3", "E5M2"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gaac-lossless-kv-cache-compression-for-faster-llm-inference", "markdown": "https://wpnews.pro/news/gaac-lossless-kv-cache-compression-for-faster-llm-inference.md", "text": "https://wpnews.pro/news/gaac-lossless-kv-cache-compression-for-faster-llm-inference.txt", "jsonld": "https://wpnews.pro/news/gaac-lossless-kv-cache-compression-for-faster-llm-inference.jsonld"}}