18:20
2026-10-12
discuss.huggingface.co
ai-infrastructure
Negative result: tail-latency gating for KV offload doesn't beat LRU+TTL (runnable simulator)
A runnable simulator published on GitHub as kv-cache-weigh-station v4.3 found that tail-quantile admission gating for HBM-to-host KV cache promotion did not beat plain LRU+TTL host retention with sign…