I Watched a Neural Network Learn That Paris Is the Capital of France
EleutherAI's Pythia-410M model, trained on The Pile corpus, shows that neural network knowledge forms through gradual accretion rather than sudden ignition, with Paris reaching 15% probability as capi…