Escaping Flatland: Weight-Space Regularization and Hyperbolic Geometry in Mechanistic Interpretability
Researchers propose Hyperbolic Sparse Autoencoders (HypSAEs) to overcome geometric limitations in mechanistic interpretability of large language models. By applying hyperbolic geometry as a weight-space regularizer, the …