# GAAC: Lossless KV Cache Compression for Faster LLM Inference

> Source: <https://arxiv.org/abs/2610.38210>
> Published: 2026-10-31 15:34:12+00:00

# Computer Science > Artificial Intelligence

  [Submitted on 28 Oct 2026]

# Title:GAAC: Lossless KV Compression for Bandwidth-Efficient LLM Inference on Personal Devices

[View PDF](https://arxiv.org/pdf/2610.38210)

[HTML (experimental)](https://arxiv.org/html/2610.38210v1)

Abstract:Weight-only, lossy quantization substantially reduces the storage and memory requirements for serving large language models (LLMs), but the growing key-value (KV) cache remains a bottleneck for long-context inference on personal devices. We present Group-Adaptive Additive Codec (GAAC), a lossless KV compression and serving system that exploits exponent redundancy to reduce storage and accelerate attention without introducing KV approximation. GAAC encodes BF16 exponents using calibrated per-channel base-plus-offset codes over contiguous ranges. GAAC adapts code widths across small groups while preserving KV values bit-for-bit. A fused attention kernel reconstructs KV tiles on the fly, avoiding full-cache decompression. Block-wise compression and fixed-capacity slots support incremental cache updates and paged serving. Across five tested models, GAAC achieves 1.405-1.407$\times$ lossless BF16 KV compression. We also study compression performance on low-precision tensors FP8 E4M3 caches ( 1.09-1.14$\times$) and E5M2 caches (1.25-1.30$\times$). On NVIDIA DGX Spark, GAAC accelerates decode attention at a 64K-token context by 1.19-1.27$\times$ and end-to-end W4A16 decode steps by 1.04-1.15$\times$. In serving, it reduces resident KV memory by approximately 26% and increases aggregate throughput by 13-29% under comparable KV-memory budgets. On Apple M4, GAAC achieves 1.02-1.25$\times$ attention speedups, demonstrating portability beyond NVIDIA GPUs.
    

### Additional Features

### References & Citations

Loading...

# Bibliographic and Citation Tools

Bibliographic Explorer 

*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))
Connected Papers 

*(*[What is Connected Papers?](https://www.connectedpapers.com/about))
Litmaps 

*(*[What is Litmaps?](https://www.litmaps.co/))
scite Smart Citations 

*(*[What are Smart Citations?](https://www.scite.ai/))
# Code, Data and Media Associated with this Article

alphaXiv 

*(*[What is alphaXiv?](https://alphaxiv.org/))
CatalyzeX Code Finder for Papers 

*(*[What is CatalyzeX?](https://www.catalyzex.com))
DagsHub 

*(*[What is DagsHub?](https://dagshub.com/))
Gotit.pub 

*(*[What is GotitPub?](http://gotit.pub/faq))
Hugging Face 

*(*[What is Huggingface?](https://huggingface.co/huggingface))
ScienceCast 

*(*[What is ScienceCast?](https://sciencecast.org/welcome))
# Demos

# Recommenders and Search Tools

Influence Flower 

*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))
CORE Recommender 

*(*[What is CORE?](https://core.ac.uk/services/recommender))
# arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).