04:00
2026-07-08
arxiv.org
large-language-models
Benchmarking KV-Cache Optimizations across Task Quality and System Performance for Long-Context Serving
A new benchmark evaluates KV-cache optimization techniques—quantization, pruning, and merging—for long-context LLM serving, finding that compression ratio alone poorly predicts end-to-end performance.…