Researchers show that incorrect L0 hyperparameter settings in Sparse Autoencoders (SAEs) cause feature mixing and degenerate solutions, preventing proper disentanglement of LLM internal features. They…
Researchers developed custom Triton kernels that accelerate JumpReLU Sparse Autoencoder inference by 2–14× on real SAEs, exploiting activation sparsity to skip zero entries during matrix multiplicatio…
Researchers at an undisclosed institution analyzed LoRA fine-tuning in Gemma-2-9B using sparse autoencoders, finding that adapter-specific feature dictionaries show weak geometric alignment with pretr…
Researchers have introduced Transcoders, a function-centric framework that decomposes vision-language models into interpretable computational pathways linking image patches to token generation. Applie…