KVBoost

mentions 2 type Organization feed RSS

// recent coverage 2 mentions

04:00

2026-08-25

arxiv.org

large-language-models

KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference

KVBoost, a chunk-level key-value cache reuse system for HuggingFace-compatible decoder models, reduces time-to-first-token by 4.49x (142.4 ms vs. 639.1 ms) on Qwen/Qwen2.5-3B over 1,000 bug-localizati…

04:47

2026-05-22

pythongiant.github.io

large-language-models

Show HN: KVBoost – chunk-level KV cache reuse for HuggingFace, 5–48x faster TTFT

KVBoost is a new open-source Python library that accelerates HuggingFace LLM inference by implementing chunk-level KV cache reuse, achieving 3–5× faster time-to-first-token (TTFT) and up to 85% cache …

// co-occurs with top 8 entities

HuggingFace 2 Qwen2.5-3B 2 FlashAttention-2 1 AWQ 1 Qwen2.5-32B 1 CPU Paged Decoding 1 Qwen 1 arXiv 1

// topics top 6 topics

large language models 2 machine learning 2 artificial intelligence 2 open source 1 developer tools 1 ai infrastructure 1