{"slug": "universality-of-gradient-descent-neural-network-training", "title": "Universality of Gradient Descent Neural Network Training", "summary": "A new paper on arXiv (2007.13664) proves a universality result for neural network training: if any algorithm can find good weights for a classification task, then an extension of the network can reproduce those weights and outputs via gradient descent alone. The authors state the construction is not intended for practical use but offers insight into meta-learning possibilities.", "body_md": "# Computer Science > Machine Learning\n\n[Submitted on 27 Jul 2020]\n\n# Title:Universality of Gradient Descent Neural Network Training\n\n[View PDF](/pdf/2007.13664)\n\n[HTML (experimental)](https://arxiv.org/html/2007.13664v1)\n\nAbstract:It has been observed that design choices of neural networks are often crucial for their successful optimization. In this article, we therefore discuss the question if it is always possible to redesign a neural network so that it trains well with gradient descent. This yields the following universality result: If, for a given network, there is any algorithm that can find good network weights for a classification task, then there exists an extension of this network that reproduces these weights and the corresponding forward output by mere gradient descent training. The construction is not intended for practical computations, but it provides some orientation on the possibilities of meta-learning and related approaches.\n\n### Current browse context:\n\ncs.LG\n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer\n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers\n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps\n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations\n\n*(*[What are Smart Citations?](https://www.scite.ai/))# Code, Data and Media Associated with this Article\n\nalphaXiv\n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers\n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub\n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub\n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face\n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast\n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower\n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender\n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\nIArxiv Recommender\n\n*(*[What is IArxiv?](https://iarxiv.org/about))# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [ Learn more about arXivLabs](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/universality-of-gradient-descent-neural-network-training", "canonical_source": "https://arxiv.org/abs/2007.13664", "published_at": "2026-08-20 00:05:12+00:00", "updated_at": "2026-08-20 00:45:09.370845+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/universality-of-gradient-descent-neural-network-training", "markdown": "https://wpnews.pro/news/universality-of-gradient-descent-neural-network-training.md", "text": "https://wpnews.pro/news/universality-of-gradient-descent-neural-network-training.txt", "jsonld": "https://wpnews.pro/news/universality-of-gradient-descent-neural-network-training.jsonld"}}