Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster
Researchers from Stanford and UC Berkeley released TokenGate, a drop-in learned token pruning module for large language models that cuts long-context prefill time-to-first-token by 1.4x to 11.3x and p…