Show HN: KVBoost – chunk-level KV cache reuse for HuggingFace, 5–48x faster TTFT
KVBoost is a new open-source Python library that accelerates HuggingFace LLM inference by implementing chunk-level KV cache reuse, achieving 3–5× faster time-to-first-token (TTFT) and up to 85% cache hit rates in multi-t…