Rajamanoharan et al

Researchers developed custom Triton kernels that accelerate JumpReLU Sparse Autoencoder inference by 2–14× on real SAEs, exploiting activation sparsity to skip zero entries during matrix multiplicatio…