04:00
2026-08-20
arxiv.org
artificial-intelligence
Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems
Researchers propose Fractional Decay KV-Cache (FD-KVC), a new algorithm for key-value cache management in transformer-based dialog systems that outperforms the state-of-the-art H2O baseline by +6.7% o…