We introduce Block-Sparse Featurizers (BSF), a family of methods to decompose a model's activations into multidimensional subspaces rather than single directions. Applied to vision models, we find that BSFs find interpretable, multidimensional features which offer a more parsimonious explanation of model internals; that those features enable fine-grained steering; and that most concepts in the models are multidimensional. *
Authors
[Aiden Swann](https://goodfire.ai)1,3
[Fenil Doshi](https://goodfire.ai)1,2
[Jack Merullo](https://goodfire.ai)†,1
* Equal contribution
† Equal senior contribution
1 Goodfire
2 Harvard University
3 Stanford University
4 Brown University
Blog post by Thomas Fel, Matthew Kowal, and Michael Byun
Picture a tree.
Now make it small, a sapling; then tall, an oak; then enormous, a redwood; then tiny again, a bonsai. Turn its leaves summer green, then autumn red, then strip them bare. Change its species: birch, pine, willow. Each one is still a tree, but they differ along many axes of variation. The concept "tree" is not a one-dimensional idea, but a multidimensional space of
variation. Conceptual Spaces: The Geometry of Thought. Trees are just one example here; our world is full of intricate structure. So are the internals of neural networks! But the most popular methods we use to understand what's happening inside neural networks, like sparse autoencoders (SAEs) and transcoders, treat each internal concept as a single straight line – an assumption which is surprisingly powerful, but which nonetheless doesn't effectively capture the naturally curved, multidimensional structures we know exist in model representations.
In this post, we explain our new paper which introduces a family of techniques – Block-Sparse Featurizers (BSF) – that we think provide a simple, viable alternative. BSFs decompose model activations into subspaces, which empirically contain multidimensional shapes.
We believe this is a significant step forward in understanding neural networks on their own terms. Applied to vision models, we find that BSFs find interpretable, multidimensional features which better explain model activations; that those features enable fine-grained steering; and that most concepts in the models are multidimensional.
Contents
Introduction: A toy model of manifolds in superposition #
As part of interpretability's broader goal of understanding what goes on inside neural networks, we want to know what a model is "thinking" at any given moment. The primary way we do thisour work on decomposing weights.
That question can be reframed as a different one: how are a model's representations structured? If we know that, then we can choose the best method to disentangle them.
If you've been following this series of posts, you can probably guess where this is going. We've discussed the growing body of research, from Goodfire as well as many others, showing that concepts are often represented inside neural networks as manifolds – multidimensional, curved shapes. We've also discussed the mismatch between these complex representations and the tools we use to find them: the most popular methods, like SAEs and transcoders, look only for straight lines.
To get an intuition for the consequences of this mismatch, consider the toy model above. We constructed clean artificial activations by summing samples from predefined manifolds – implicitly stacking the manifolds in superposition. This forms a simplified model of how we think real neural network activations are structured. Then we use two different featurizer methods to try to recover the known underlying structure from the activations.
On the left are the ground-truth manifolds, which are what we're aiming to
recover. In the middle column, we see that a standard SAE does a poor job of
finding the underlying manifolds, we showed SAEs don't lose manifold geometry so much as fragment it by tiling manifolds across many redundant features that only post-hoc analysis (with an Ising-model and extra compute) can regroup. In contrast, BSFs recover these subspaces coherently during training, even when the SAE is given the advantage of selecting its best top-g directions against a single BSF block of size g.
But if we instead begin with the assumption that model activations are made of manifolds, that implies a different strategy, whose results are on the right. A Block-Sparse Featurizer (BSF) – the method this post describes – finds the ground-truth structures with much higher fidelity!
So: how do BSFs work, and why are they effective at finding manifolds?
Block-Sparse Featurizers (BSFs) #
We want a method that respects neural geometry. So instead of looking for one-dimensional lines as the fundamental "atoms" of neural computations, we look for multidimensional subspaces. Since each subspace is multidimensional, it can far better capture the shape taken by a concept's representations.
How do we disentangle these subspaces? The key insight is that a block – the set of directions that span a subspace – should tend to "fire" together on the same input. Unlike SAEs, which assume that a small (sparse) number of 1D features are firing at any given time, BSFs assume that a small (sparse) number of blocks are firing at any given time. That is, we enforce sparsity at the block level.
This principle is what we call block-sparsity, and it is not new: it belongs to the older family of structured sparsity, a long-studied topic in dictionary learningYuan & Lin, 2006; Eldar & Bolksei, 2009; Jenatton et al., 2009.
In the course of this research, we found that the principle of block-sparsity matters more than the particular implementation. We built several methods that follow the general principle, yet differ in how they impose that structure,
The diagram below shows the architecture of a generic BSF:
We propose three different BSF variants: vanilla BSF, Grassmannian BSF, and group-lasso BSF. All three share the generic architecture in the diagram above, but they differ in their choice of encoder/decoder and activation function. For example, the vanilla BSF uses a standard linear encoder $\mathbf{xW} + \mathbf{b}$ and a block-wise TopK (based on the L2 norm of each block) as its activation function. All BSF methods we propose are trained to minimize reconstruction error. For more details on the three variants and their theoretical grounding, see the
What BSFs find in vision models #
The real test of any featurizer is the quality of the features that it recovers from models that are actually used in production: do the features correspond to coherent concepts? How well do they explain the model's activations? And what do the features allow you to do with the model?
We trained BSFs on two off-the-shelf open-source models: a vision model, DINOv3, and a text-to-image model, Stable Diffusion XL (SDXL). Unlike in the toy model from earlier, the structure of the model's representations is not known in advance.
Quantitatively, we find that BSFs yield a more faithful and efficient account of the model: they better explain the original model activations with a lower minimum description length.
Visual concept manifolds
In these vision models, the activations of a BSF block rarely form a line, instead forming dense, connected clouds. A standard SAE squashes each cloud down to a single direction, shown as the black lines in the figure below. A BSF keeps the whole cloud instead, treating that entire region as the concept.
This shows up clearly in DINOv3. Take the "arch feature" below: the BSF finds a region with interpretable internal structure. Moving through the region traces out the different parts of an arch. Whether in a viaduct or a church's nave arcade, one part of the region (visualized in red) activates on the bottom of an arch, smoothly transitioning to another part of the region (visualized in green) which activates across the top of an arch.
Generating images with manifold steering
Because a BSF block gives us a region of activations corresponding to a concept, rather than a single point, we can move around inside it. This lets us steer along the concept, sweeping through its internal variations rather than turning a one-dimensional knob up or down.
We can see this directly by laying a grid over the region and walking from point to point. Each step moves us through the range of examples that belong to one concept, so we can explore the space and watch the many kinds of pretzels the model can draw: the "pretzel manifold".
Below, we show two more examples, simplifying the multidimensional structure into a 2D grid. We can see high-level concepts varying smoothly across tiles: the hat's brim angle, crown shape, and orientation, or the coffee's milk pattern, spoon location, or cup design.
Curve detectors in InceptionV1: Revisiting an old friend
We generate images with curves at different angles to study InceptionV1.
(a) We show how neurons, SAE features, and blocks activate as the angle of a curve changes. Neurons and SAE atoms provide a fractured view of the unified, continuous structure captured in a block.(b) Different components of the block respond to the rotation of curves at different rates.(c) A visualization of how each harmonic responds to rotating curves.We now revisit a feature the interpretability community has studied for years, using BSFs to examine it from a new angle.
Curve detectors in InceptionV1 are a classic case study in interpretability. Cammarata et al. (2020)Cammarata et al., "Curve Detectors," Distill (2020).Gorton, "The Missing Curve Detectors of InceptionV1" (2024).
Using our BSFs, we showed that both the neurons and the SAE features actually give us a fragmented view of a continuous space that smoothly captures the rotation of curves! But it goes deeper than that: beyond recovering orientation as a continuum in a single subspace, the BSF also surfaces other features in that same block, which turn out to be higher-order Fourier harmonics - a property of the representation that has not been documented before. These harmonics are symmetries the model has learned: the first harmonic encodes the full 360 degrees of possible orientations (peaking at 0°), the second activates for curves that are the same when flipped around (peaking at 0° and 180°), and so on for the higher harmonics.
This also explains an observation from Olah et al. (2020)Olah et al., "An Overview of Early Vision in InceptionV1," Distill (2020).
How common are multidimensional features?
With a featurizer that looks for subspaces rather than single directions, we can ask a question that was hard to even approach before: how often are concepts actually multidimensional in neural network activations?
Once the featurizer is trained, we can study the stable rank of each block – the effective number of dimensions it uses – as a measure of how fully that block is exploited. If we give the model blocks of size 16, it could in principle use all 16 dimensions equally. In practice, though, we find that a block rarely exceeds a stable rank of 4. This is perhaps unsurprising: visual concepts are 2D projections of a three-dimensional scene.
In vision models, at least, most concepts appear to be multidimensional – and
specifically, two- to four-dimensional
Looking forward #
Our earlier work kept finding concepts with rich, multidimensional structure, which our tools struggled to capture. So we adjusted the tools to match the geometry we were seeing. The new featurizer does what it was designed to do, recovering most concepts as multidimensional representations. We believe the principle of block-sparsity matters more than any single way we implement it, and we expect sharper implementations of it to come.
Other settings will likely call for their own variants. For example, vision-language-action models (used in robotics) lay their activations out along a temporal trajectory. Each domain may need a featurizer shaped to its own geometry, and block-sparsity is unlikely to be the right prior everywhere.
That is really the broader point. A featurizer is a hypothesis about how a model's representations are structured, and part of the job of interpretability is to work out what those building blocks should be in the first place. Block-Sparse Featurizers are one answer to this question, and one step in a longer agenda of reading the geometry of a model and letting it tell us what the next answer should be.
For prior work, theoretical grounding, more experiments, and details on training BSFs, read the full paper. Our code to train BSFs is available on GitHub.