Learning-Induced Dynamical Transition in Recurrent Neural Networks

A non-equilibrium dynamical mean-field theory (DMFT) developed by researchers describes how learning in recurrent neural networks drives a transition from chaotic to stable task-dependent dynamics, according to a paper published as arXiv:2609.19288v1. The theory identifies a critical feedback strength and a learning-rate-dependent critical time that separate the chaotic and stable regimes, derived from the two-time correlation function throughout learning. The authors report that the theory predicts the time evolution of network output during training and shows quantitative agreement with numerical simulations.

arXiv:2609.19288v1 Announce Type: new Abstract: Learning in recurrent neural networks can fundamentally reshape their underlying dynamics, transforming initially chaotic activity into stable task-dependent behavior. We develop a non-equilibrium dynamical mean-field theory DMFT to describe this transition during learning. We show that a slow feedback-driven learning process generates an evolving effective feedback strength that drives the network through a transition from chaotic to stable dynamics defined by a bifurcation of the DMFT solution. By deriving the two-time correlation function throughout learning, we identify a critical feedback strength and a corresponding learning rate dependent critical time separating these regimes. The transition arises from the progressive deformation of an effective dynamical landscape by the growing learned feedback structure. Starting from the untrained state, the theory predicts the time evolution of the network output during training and shows quantitative agreement with numerical simulations.