Negative result: tail-latency gating for KV offload doesn't beat LRU+TTL (runnable simulator)

Runnable simulator for a KV offload negative result: GitHub - MrBeeboh/kv-cache-weigh-station: Negative result: tail-gated KV admission vs LRU+TTL host retention — protocol, simulator, kill criteria · GitHub (./reproduce.sh, about 2 min, Mooncake traces). I tested tail-quantile admission gating for HBM↔host KV promotion against plain LRU+TTL host retention with signal prefetch. The gate did not win: −0.0% goodput, 95% CI [−2.3, +1.1]. Under predictive prefetch, a simple p_use > θ threshold matched it. My next step is a single-GPU measurement of decode slowdown under concurrent KV DMA. If anyone has numbers for cudaMemcpyAsync versus gather-kernel copies during decode, I’d appreciate a pointer. The protocol and kill criteria are in the PDF (release v4.3): Release v4.3 · MrBeeboh/kv-cache-weigh-station · GitHub

Disclosure: AI assistants helped draft the paper and write the simulator; they are named in the paper.