How are linear representations learned? Exact solutions to the dynamics of abstraction
Researchers derived exact solutions for how linear representations (concept directions) emerge during neural network training, a process they call abstraction. Their theory reveals that data and targe…