{"slug": "pre-warm-input-conditioned-weight-initialization-for-convolutional-neural", "title": "Pre-Warm: Input-Conditioned Weight Initialization for Convolutional Neural Networks", "summary": "Researchers introduced Pre-Warm, a zero-training-cost method for data-conditioned initialization of the first convolutional layer in CNNs. The technique clusters patches from a single training batch to initialize half of the first-layer filters, yielding statistically significant accuracy improvements over standard Kaiming initialization across five benchmarks. Pre-Warm adds negligible overhead and requires no architectural changes.", "body_md": "arXiv:2606.25256v1 Announce Type: new\nAbstract: We introduce Pre-Warm, a simple yet effective zero-training-cost method for data-conditioned initialization of the first convolutional layer. Before the first forward pass, Pre-Warm extracts mean-centered local patches from a single training batch, clusters them with MiniBatchKMeans, applies inverse Manhattan spatial weighting, and uses the resulting centroids to initialize half of the first-layer filters (the remainder retain Kaiming initialization).\nWe derive closed-form rules for all hyperparameters except a single insensitive scale parameter, though we derive a Kaiming parity bound on scale from patch dimensionality. For grayscale datasets we use Otsu's foreground density; for natural color images we use the mean L2 norm of mean-centered patches. Both rules accurately predict the optimal patch count observed in grid search.\nAcross five standard benchmarks -- MNIST, Fashion-MNIST, CIFAR-10, SVHN, and CIFAR-100 -- and 8-seed paired experiments, Pre-Warm yields statistically significant accuracy improvements over standard Kaiming initialization (p < 0.05 on all datasets, p = 0.0007 on SVHN with 8/8 wins, p = 0.0033 on CIFAR-100 with 7/8 wins). The method adds negligible overhead, requires no architectural changes, and integrates into existing training pipelines with only a few lines of code.\nPre-Warm demonstrates that even a lightweight, input-dependent signal can meaningfully improve optimization trajectories in modern convolutional networks.", "url": "https://wpnews.pro/news/pre-warm-input-conditioned-weight-initialization-for-convolutional-neural", "canonical_source": "https://arxiv.org/abs/2606.25256", "published_at": "2026-06-25 04:00:00+00:00", "updated_at": "2026-06-25 04:20:17.428631+00:00", "lang": "en", "topics": ["machine-learning", "computer-vision", "neural-networks"], "entities": ["Pre-Warm", "MiniBatchKMeans", "Kaiming initialization", "MNIST", "Fashion-MNIST", "CIFAR-10", "SVHN", "CIFAR-100"], "alternates": {"html": "https://wpnews.pro/news/pre-warm-input-conditioned-weight-initialization-for-convolutional-neural", "markdown": "https://wpnews.pro/news/pre-warm-input-conditioned-weight-initialization-for-convolutional-neural.md", "text": "https://wpnews.pro/news/pre-warm-input-conditioned-weight-initialization-for-convolutional-neural.txt", "jsonld": "https://wpnews.pro/news/pre-warm-input-conditioned-weight-initialization-for-convolutional-neural.jsonld"}}