H2O

mentions 2 type Organization feed RSS

// recent coverage 2 mentions

04:00

2026-08-20

arxiv.org

artificial-intelligence

Fractional Decay KV-Cache: Ownership-Aware Memory Management for Improved Inference Relevancy in Dialog Systems

Researchers propose Fractional Decay KV-Cache (FD-KVC), a new algorithm for key-value cache management in transformer-based dialog systems that outperforms the state-of-the-art H2O baseline by +6.7% o…

08:18

2026-08-10

ssenthilnathan3.github.io

machine-learning

MoE routing is just branch prediction

A software engineer's analysis argues that MoE routing in transformer inference is fundamentally the same problem as CPU branch prediction, and that KV cache management techniques such as prefix cachi…

// co-occurs with top 6 entities

Fractional Decay KV-Cache 1 MoE 1 KV cache 1 vLLM 1 SGLang 1 StreamingLLM 1

// topics top 5 topics

artificial intelligence 2 machine learning 2 large language models 1 ai research 1 ai infrastructure 1