{"slug": "replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under", "title": "Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model", "summary": "Researchers developed an incremental approximate message passing (IAMP) algorithm for empirical risk minimization under a multi-index model, achieving near-optimal training and test error in high-dimensional asymptotics. The work provides a precise characterization of which parts of the non-convex loss landscape are accessible by polynomial-time algorithms, with implications for understanding why gradient-based optimization succeeds in modern machine learning.", "body_md": "arXiv:2606.28573v1 Announce Type: new\nAbstract: Modern machine learning models are trained by optimizing high-dimensional non-convex empirical risk functions. Such cost functions can have a multitude of local optima and yet, gradient-based optimization appears to converge to near-global optima.\nWithin a simple supervised learning setting, we develop a precise picture of which parts of the empirical risk landscape are accessible by polynomial-time algorithms. We are given i.i.d. pairs $\\{(\\boldsymbol{x}_i,y_i):\\; 1 \\le i\\le n\\}$ with $\\boldsymbol{x}_i\\in \\mathbb{R}^d$ standard Gaussian feature vectors, and $y_i\\in\\mathbb{R}$ response variables that depend on $\\boldsymbol{x}_i$ through their projections on an unknown $k$-dimensional subspace. We use empirical risk minimization to learn a model that depends on an $m$-dimensional projection of the data (e.g., an $m$-neurons neural network).\nWe propose an incremental approximate message passing (IAMP) algorithm and precisely characterize the training error it achieves, as well as the relation between test and training error, in the high dimensional asymptotics $n,d\\to\\infty$, with $n/d\\to\\alpha \\in (0, +\\infty)$. Based on earlier work in related models, we expect that the performance achieved by our algorithm is optimal among polynomial-time algorithms.", "url": "https://wpnews.pro/news/replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under", "canonical_source": "https://arxiv.org/abs/2606.28573", "published_at": "2026-06-30 04:00:00+00:00", "updated_at": "2026-06-30 04:30:17.463759+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under", "markdown": "https://wpnews.pro/news/replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under.md", "text": "https://wpnews.pro/news/replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under.txt", "jsonld": "https://wpnews.pro/news/replica-symmetry-breaking-and-algorithmic-thresholds-in-empirical-risk-under.jsonld"}}