{"slug": "orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics", "title": "Orthogonal Gradient Constraints Shape Noisy-Label Memorization Dynamics", "summary": "A new study on arXiv (2607.16231v1) evaluates OrthoGrad, a geometric intervention that removes the radial component of weight gradients during optimizer updates, in noisy-label image classification. On MNIST with small-data regimes, OrthoGrad improves test accuracy for CNNs while reducing corrupted-label fitting, but on CIFAR-10 with ResNet-18 it alters memorization trajectories without preventing eventual noisy-label memorization. The findings show that orthogonal update constraints are regime-dependent and serve as a useful diagnostic for studying learning dynamics.", "body_md": "arXiv:2607.16231v1 Announce Type: new\nAbstract: Modern neural networks can fit corrupted training labels, making noisy-label learning a useful setting for studying memorization-driven overfitting. Most regularization methods modify the objective, architecture, or data distribution; here we instead study a geometric intervention on the optimizer update itself. We evaluate OrthoGrad, which removes the component of each weight gradient parallel to the current weight vector, in noisy-label image classification. On MNIST with small-data regimes, OrthoGrad improves test accuracy most clearly for CNNs while reducing corrupted-label fitting. Mechanism diagnostics based on weight norms and gradient-weight cosine similarity suggest that the projection has the strongest effect when the raw gradient contains a nontrivial radial component, and becomes weaker in larger-data regimes where gradients are already nearly orthogonal to weights. Additional CIFAR-10 ResNet-18 experiments show that the method can alter memorization trajectories but does not prevent eventual noisy-label memorization. These results support orthogonal update constraints as a useful diagnostic for studying learning dynamics, while showing that OrthoGrad is regime-dependent rather than universally regularizing.", "url": "https://wpnews.pro/news/orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics", "canonical_source": "https://arxiv.org/abs/2607.16231", "published_at": "2026-07-21 04:00:00+00:00", "updated_at": "2026-07-21 04:12:13.995735+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence"], "entities": ["arXiv", "OrthoGrad", "MNIST", "CIFAR-10", "ResNet-18"], "alternates": {"html": "https://wpnews.pro/news/orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics", "markdown": "https://wpnews.pro/news/orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics.md", "text": "https://wpnews.pro/news/orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics.txt", "jsonld": "https://wpnews.pro/news/orthogonal-gradient-constraints-shape-noisy-label-memorization-dynamics.jsonld"}}