# Negative result: tail-latency gating for KV offload doesn't beat LRU+TTL (runnable simulator)

> Source: <https://discuss.huggingface.co/t/negative-result-tail-latency-gating-for-kv-offload-doesnt-beat-lru-ttl-runnable-simulator/190421#post_1>
> Published: 2026-10-12 18:20:14+00:00

Runnable simulator for a KV offload negative result: [GitHub - MrBeeboh/kv-cache-weigh-station: Negative result: tail-gated KV admission vs LRU+TTL host retention — protocol, simulator, kill criteria · GitHub](https://github.com/MrBeeboh/kv-cache-weigh-station) (`./reproduce.sh`, about 2 min, Mooncake traces). I tested tail-quantile admission gating for HBM↔host KV promotion against plain LRU+TTL host retention with signal prefetch. The gate did not win: −0.0% goodput, 95% CI [−2.3, +1.1]. Under predictive prefetch, a simple p_use > θ threshold matched it. My next step is a single-GPU measurement of decode slowdown under concurrent KV DMA. If anyone has numbers for cudaMemcpyAsync versus gather-kernel copies during decode, I’d appreciate a pointer. The protocol and kill criteria are in the PDF (release v4.3): [Release v4.3 · MrBeeboh/kv-cache-weigh-station · GitHub](https://github.com/MrBeeboh/kv-cache-weigh-station/releases/tag/v4.3)

*Disclosure: AI assistants helped draft the paper and write the simulator; they are named in the paper.*