BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

Researchers propose BeaconKV, a KV cache compression method for Large Reasoning Models (LRMs) that uses beacon queries to guide compression, addressing memory bottlenecks from long Chain-of-Thought generation. The method aims to reduce GPU memory usage while maintaining model performance.

Large Reasoning Models LRMs achieve superior problem-solving through extended Chain-of-Thought CoT generation, but the resulting key-value KV cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cac