Uncertainty Estimation and Generalization Bounds for Modern Deep Learning

A new thesis investigates how Bayesian principles can improve understanding of modern deep learning systems, introducing the Deep Variational Implicit Process (DVIP) and post-hoc methods VaLLA and FMGP for uncertainty estimation. The theoretical work develops a unified probabilistic framework connecting diversity, smoothness, and stochasticity to explain why over-parameterized neural networks generalize well.

arXiv:2606.13818v1 Announce Type: new Abstract: This thesis investigates how Bayesian principles can deepen our understanding of modern deep learning systems. While neural networks achieve remarkable predictive performance, their ability to generalize and to quantify uncertainty remains only partly understood. This thesis approaches this challenge from both methodological and theoretical angles: unifying Bayesian inference, function-space modeling, and large-deviation theory under a common probabilistic perspective. On the methodological side, the thesis introduces the Deep Variational Implicit Process DVIP , a scalable Bayesian framework that extends implicit processes to deep architectures. Complementing this, two post-hoc methods -- the Variational Linearized Laplace Approximation VaLLA and the Fixed-Mean Gaussian Process FMGP -- are proposed to equip pretrained deterministic networks with calibrated uncertainty estimates. The theoretical contributions focus on one of the central open questions in modern machine learning: why do large, over-parameterized neural networks generalize so well? To address this, the thesis develops a unified probabilistic framework that connects three key mechanisms -- diversity, smoothness, and stochasticity -- within the language of PAC-Bayesian and large-deviation theory.