arXiv:2611.24969v1 Announce Type: new Abstract: Memory consumption of the key-value (KV) cache remains a central bottleneck for long-context LLM inference. We introduce PRKV, a training-free KV eviction method that estimates multi-hop global token importance by applying Personalized PageRank over attention-induced transition graphs. This enables importance propagation beyond recency or instantaneous attention, without modifying the model. We benchmark PRKV on long-context understanding, sparse retrieval, and chain-of-thought reasoning tasks using multiple LLM families. Under moderate to aggressive compression ratios, including up to 90%, PRKV achieves competitive task performance against established eviction baselines while reducing peak memory usage with low end-to-end overhead. These results demonstrate that multi-hop global influence estimation yields a favorable accuracy--memory trade-off for scalable long-context inference.
Global Importance Estimation for KV Cache Eviction
A new arXiv paper (2611.24969v1) introduces PRKV, a training-free KV cache eviction method that applies Personalized PageRank over attention-induced transition graphs to estimate multi-hop global token importance for long-context LLM inference. Benchmarked on long-context understanding, sparse retrieval, and chain-of-thought reasoning across multiple LLM families, PRKV remained competitive with established eviction baselines at compression ratios up to 90% while cutting peak memory usage with low end-to-end overhead. The authors say the results show multi-hop global influence estimation delivers a favorable accuracy-memory trade-off for scalable long-context inference without modifying the model.
When KV cache memory caps long-context inference, evict tokens by multi-hop Personalized PageRank importance over the attention graph rather than recency or instantaneous attention; the authors held competitive accuracy at up to 90% compression.
more in playbook: Inference and self-hostingRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.