GAAC: Lossless KV Cache Compression for Faster LLM Inference

A paper submitted to arXiv on 28 Oct 2026 presents Group-Adaptive Additive Codec (GAAC), a lossless KV cache compression and serving system that achieves 1.405-1.407x BF16 KV compression across five tested models while preserving KV values bit-for-bit. On NVIDIA DGX Spark, GAAC speeds decode attention at a 64K-token context by 1.19-1.27x and end-to-end W4A16 decode steps by 1.04-1.15x, cuts resident KV memory by roughly 26% and raises aggregate throughput 13-29% under comparable KV-memory budgets; on Apple M4 it delivers 1.02-1.25x attention speedups. GAAC encodes BF16 exponents with calibrated per-channel base-plus-offset codes and uses a fused attention kernel that reconstructs KV tiles on the fly, avoiding full-cache decompression.

Computer Science Artificial Intelligence Submitted on 28 Oct 2026 Title:GAAC: Lossless KV Compression for Bandwidth-Efficient LLM Inference on Personal Devices View PDF https://arxiv.org/pdf/2610.38210 HTML experimental https://arxiv.org/html/2610.38210v1 Abstract:Weight-only, lossy quantization substantially reduces the storage and memory requirements for serving large language models LLMs , but the growing key-value KV cache remains a bottleneck for long-context inference on personal devices. We present Group-Adaptive Additive Codec GAAC , a lossless KV compression and serving system that exploits exponent redundancy to reduce storage and accelerate attention without introducing KV approximation. GAAC encodes BF16 exponents using calibrated per-channel base-plus-offset codes over contiguous ranges. GAAC adapts code widths across small groups while preserving KV values bit-for-bit. A fused attention kernel reconstructs KV tiles on the fly, avoiding full-cache decompression. Block-wise compression and fixed-capacity slots support incremental cache updates and paged serving. Across five tested models, GAAC achieves 1.405-1.407$\times$ lossless BF16 KV compression. We also study compression performance on low-precision tensors FP8 E4M3 caches 1.09-1.14$\times$ and E5M2 caches 1.25-1.30$\times$ . On NVIDIA DGX Spark, GAAC accelerates decode attention at a 64K-token context by 1.19-1.27$\times$ and end-to-end W4A16 decode steps by 1.04-1.15$\times$. In serving, it reduces resident KV memory by approximately 26% and increases aggregate throughput by 13-29% under comparable KV-memory budgets. On Apple M4, GAAC achieves 1.02-1.25$\times$ attention speedups, demonstrating portability beyond NVIDIA GPUs. Additional Features References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .