{"slug": "pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent", "title": "Pseudospectral Bounds for Transient Amplification in Coupled Gradient Descent", "summary": "A new pseudospectral theory for block-triangular Jacobians in coupled gradient descent proves that transient amplification before convergence can be arbitrarily large even when asymptotic stability is guaranteed, with the Kreiss constant bounded by \\(2/(1-\\gamma) + \\|C\\|/(4(1-\\gamma))\\) for symmetric diagonal blocks. The analysis establishes matching minimax lower bounds and a finite-horizon iteration-complexity bound of \\(O(K(J)^2 \\log(1/\\delta))\\) for stochastic coupled descent, exposing a non-asymptotic, instance-dependent regime invisible to spectral-radius analysis. Experiments on linear-quadratic problems and neural-network training confirm the theory, which frames the results as scaling laws for non-stationary two-time-scale optimization.", "body_md": "arXiv:2606.04031v1 Announce Type: new\nAbstract: Coupled gradient descent--where the update of one parameter block depends on another--underlies bilevel optimization, two-time-scale stochastic approximation, and adversarial training. When the coupled Jacobian is block-triangular, asymptotic stability is governed by the spectral radii of the diagonal blocks, yet transient amplification before convergence can be arbitrarily large due to non-normality. We develop a sharp pseudospectral theory for such block-triangular Jacobians, proving that the Kreiss constant satisfies $K(J) \\leq 2/(1-\\gamma) + \\|C\\|/(4(1-\\gamma))$ when the diagonal blocks are symmetric with spectral radii at most $\\gamma < 1$, and we establish matching minimax lower bounds. We characterize the critical coupling threshold for spectral instability and extend the analysis to nearly self-referential systems via a Neumann-series perturbation framework. As a consequence, we obtain a finite-horizon iteration-complexity bound of $O(K(J)^2 \\log(1/\\delta))$ for stochastic coupled descent. Framed as scaling laws for non-stationary two-time-scale optimization, our results expose a non-asymptotic, instance-dependent regime of high-dimensional learning dynamics that is invisible to spectral-radius analysis. Experiments on linear-quadratic problems, IQC-based comparisons, and neural-network training confirm the theory.", "url": "https://wpnews.pro/news/pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent", "canonical_source": "https://arxiv.org/abs/2606.04031", "published_at": "2026-06-04 04:00:00+00:00", "updated_at": "2026-06-04 04:36:26.674511+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent", "markdown": "https://wpnews.pro/news/pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent.md", "text": "https://wpnews.pro/news/pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent.txt", "jsonld": "https://wpnews.pro/news/pseudospectral-bounds-for-transient-amplification-in-coupled-gradient-descent.jsonld"}}