Information-Theoretic Design of Sparse Autoencoder-Based Steering

A new arXiv paper (2611.01378v1) introduces an information-theoretic framework for designing sparse autoencoder (SAE)-based activation steering in large language models, using Jensen-Shannon divergence (JSD) between empirical feature-firing distributions to quantify how far a generated or intervened state deviates from a dataset-induced target distribution. The authors derive an analytic divergence response for virtually shifted states to construct steering vectors toward the target firing distribution, and show the same divergence can monitor deviation from other empirical distributions such as a general valid-language distribution. They demonstrate the steering in a question-mode generation task and use teacher-forced pulse experiments to show local interventions induce structured downstream JSD responses in later layers, applying the framework to a safety-related target-contrast response setting for feature analysis.

arXiv:2611.01378v1 Announce Type: new Abstract: Sparse autoencoder SAE -based activation steering is a lightweight approach for modifying the behavior of large language models LLMs by adding small vectors to hidden states during inference. However, designing effective steering interventions remains challenging. In this work, we develop an information-theoretic framework for analyzing SAE activations through empirical feature-firing distributions. The central object of the framework is the Jensen--Shannon divergence JSD between such firing distributions, which quantifies how far a generated or intervened state deviates from a dataset-induced target distribution. Based on an analytic divergence response for virtually shifted states, we construct steering vectors toward the target firing distribution. The same divergence can also be used to monitor deviation from another empirical distribution, such as a general valid-language distribution. We demonstrate the effectiveness of the steering in a question-mode generation task study. We further use teacher-forced pulse experiments to show that local interventions induce structured downstream JSD responses in later layers, and apply the framework to a safety-related target--contrast response setting for feature analysis.