Neural Networks Are Already Clean
Mechanistic interpretability researcher Chris Olah's collaborator reports that the canonical evidence for polysemanticity and superposition in neural networks was misread, and that what looked like arbitrary concept pack…