Emergence of Fibrations, Compression, and Symmetry Breaking in Artificial Neural Networks

A new study posted on arXiv (2609.01768v1) demonstrates that learning in deep neural networks generates local symmetries known as fibrations and coverings, proving that covering symmetries are stable attractors of stochastic gradient descent. The researchers report the emergence of these symmetries across multilayer, convolutional, recurrent, and transformer networks, enabling model compression to 17% of original size without performance loss, and controlled symmetry breaking that achieves state-of-the-art continual learning results.

arXiv:2609.01768v1 Announce Type: new Abstract: Artificial neural networks are often regarded as powerful yet opaque black boxes. Here, we demonstrate that learning in deep neural networks generates local symmetries known in graph theory as fibrations and coverings. We prove that covering symmetries are stable attractors of stochastic gradient descent. Consistent with this theory, we report the emergence of covering symmetries across major network architectures, including multilayer, convolutional, recurrent, and transformer networks. Exploiting these symmetries enables drastic model compression - reducing networks to 17% of their original size without sacrificing performance. Furthermore, controlled breaking of covering symmetry overcomes the loss of plasticity, achieving state-of-the-art performance in continual learning. The theoretical results provide a new foundation for AI systems based on symmetries that convert black boxes into interpretable colored graphs and enable more efficient inference and lifelong learning.