RadixAttention

SGLang, an open-source LLM inference engine, introduces RadixAttention for prefix caching and grammar-constrained decoding to achieve 25x throughput improvement over vLLM for structured output tasks. …

Trellis introduced RadixAttention, a radix-tree-based KV cache designed to accelerate the prefill phase of LLM inference for chat and agentic sessions. The system stores shared string prefixes compact…