SnapKV

Anthropic's Jacobian lens, introduced in a July 6 paper, reveals a low-dimensional 'J-space' inside language models that encodes tokens the model can verbalize. Developer Anyesh repurposed this signal…

VarRate, a training-free variable-rate KV cache compression method for long-context LLMs, outperforms token-selection methods by allocating low-rank budgets based on query salience instead of evicting…

A new benchmark evaluates KV-cache optimization techniques—quantization, pruning, and merging—for long-context LLM serving, finding that compression ratio alone poorly predicts end-to-end performance.…