{"slug": "approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence", "title": "Approximation Property of Dropout Neural Networks: Sobolev Rates and Confidence Bounds", "summary": "A new arXiv paper (2610.02253v1) establishes approximation rates for dropout ReLU networks, showing that the unit ball of W^{n,∞}([0,1]^d) can be approximated uniformly by constant-depth networks of size Õ_{n,d}(p^{-9}ε^{-max{d/n,2}} log(1/δ)) with probability at least 1-δ for a single sampled network. The authors prove matching upper and lower bounds in the accuracy exponent for fixed retention probability p in (0,1) and δ < min{1/2, 1-p} under a fixed or logarithmic depth budget, with the bounds also matching in confidence up to logarithmic factors in accuracy when d ≤ 2n. The work leaves the optimal retention dependence and logarithmic factors open.", "body_md": "arXiv:2610.02253v1 Announce Type: new \nAbstract: The universal approximation property of dropout neural networks does not by itself describe the network size required for an accurate random realization. In this work, we study approximation of the unit ball of $W^{n,\\infty}([0,1]^d)$ by ReLU networks whose edges are retained independently with probability $p$. The approximation error is measured uniformly over the input domain, and the guarantee holds with probability at least $1-\\delta$ for a single sampled network. We construct networks of constant depth and size $\\widetilde O_{n,d}(p^{-9}\\varepsilon^{-\\max\\{d/n,2\\}} \\log(1/\\delta))$. The construction combines bounded local subnetworks, localization on a successful approximation event, and a multiscale Taylor decomposition. Conversely, Sobolev capacity imposes a lower bound on the number of surviving edges, while approximation of a fixed affine function requires an output-layer cost of order $((1-p)/p)\\varepsilon^{-2}\\log(1/\\delta)$ at sufficiently high confidence. For fixed $p\\in(0,1)$ and $\\delta<\\min\\{1/2,1-p\\}$, the upper and lower bounds match in the accuracy exponent under a fixed or logarithmic depth budget. When $d\\leq2n$, they also match in confidence up to logarithms of accuracy. We extend the lower bounds to $W^{n,r}$ targets with $L^s$ error, and distinguish this extension from the upper bound for $W^{n,\\infty}$. The optimal retention dependence and logarithmic factors remain open.", "url": "https://wpnews.pro/news/approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence", "canonical_source": "https://arxiv.org/abs/2610.02253", "published_at": "2026-10-05 04:00:00+00:00", "updated_at": "2026-10-05 04:12:28.502791+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research"], "entities": ["arXiv", "ReLU networks", "W^{n,∞}([0,1]^d)"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence", "markdown": "https://wpnews.pro/news/approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence.md", "text": "https://wpnews.pro/news/approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence.txt", "jsonld": "https://wpnews.pro/news/approximation-property-of-dropout-neural-networks-sobolev-rates-and-confidence.jsonld"}}