Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Researchers introduced Adamas, a token-level sparse attention mechanism that applies the Hadamard transform, bucketization and 2-bit compression to select which key-value tokens to attend to during long-context inference…