LRU

A September 24, 2026 arXiv paper studying production traces from two companies found that 14 eviction algorithms tested for LLM prefix caching deliver little benefit over LRU despite a large gap to Be…

In their OSDI '26 paper, researchers propose Learning-Augmented Heuristics (LAH), a cache eviction design that keeps a simple heuristic on the request path and uses machine learning to periodically co…