Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution

Researchers introduced turn-averaged sparse autoencoders (SAEs) that represent entire conversational turns with fixed feature sets, improving interpretability of long-context language models. The method outperforms per-token SAEs in capturing high-level turn characteristics and simplifies downstream tasks like attribution graphs.

arXiv:2606.28548v1 Announce Type: new Abstract: Sparse autoencoders SAEs have become a useful tool for extracting interpretable features in language models. However, standard SAE architectures operate on individual token activations, meaning that the number of active features scales linearly with context length, and studying long model transcripts becomes difficult. We introduce turn-averaged SAEs, which represent a single Human or Assistant turn with a fixed number of features by learning to reconstruct the average model activation across the turn. We find that turn-averaged features describe a single turn's high-level characteristics more completely than per-token features when judged by an LLM. We also demonstrate that turn-averaged SAEs greatly simplify common downstream uses of SAEs like attribution graphs. Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.