{"slug": "ngn-learning-neural-network-size-as-a-differentiable-count", "title": "NGN: Learning Neural Network Size as a Differentiable Count", "summary": "Researchers introduced the Neurogenesis Network (NGN), a differentiable parameterization that learns how many ordered structural components a model should use by training a single learnable boundary per component group that selects an active prefix, according to the arXiv paper 2609.27291v1. In controlled experiments the learned boundary converged, and deploying only the learned prefix usually changed performance little, with selected architectures performing similarly to fixed models trained at the same size. The mechanism was applied to MLPs, convolutional and graph networks, Transformers, state-space models, LoRA, and adapters, indicating structural capacity can be optimized directly as a count.", "body_md": "arXiv:2609.27291v1 Announce Type: new \nAbstract: Neural network size is usually chosen before training, separating architecture selection from weight optimization. We introduce the Neurogenesis Network (NGN), a differentiable parameterization for learning how many ordered structural components a model should use. For each ordered component group, one learnable boundary selects an active prefix while the model parameters are trained. The boundary can grow from a compact initialization and can be deployed by discarding components beyond the learned boundary. Controlled experiments examine convergence of the learned boundary, the performance of deployed prefixes, and comparisons with fixed-size models and alternative approaches to learning capacity. We then apply the same mechanism to MLPs, convolutional and graph networks, Transformers, state-space models, LoRA, and adapters. Across these settings, deploying only the learned prefix usually changes performance little, and the selected architectures perform similarly to fixed models trained at the same size. These results show that structural capacity can be optimized directly as a count.", "url": "https://wpnews.pro/news/ngn-learning-neural-network-size-as-a-differentiable-count", "canonical_source": "https://www.machinebrief.com/news/ngn-learning-neural-network-size-as-a-differentiable-count-8b4z", "published_at": "2026-09-24 04:00:00+00:00", "updated_at": "2026-09-24 05:00:24.862623+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "ai-research", "large-language-models"], "entities": ["Neurogenesis Network", "NGN", "arXiv", "LoRA"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ngn-learning-neural-network-size-as-a-differentiable-count", "markdown": "https://wpnews.pro/news/ngn-learning-neural-network-size-as-a-differentiable-count.md", "text": "https://wpnews.pro/news/ngn-learning-neural-network-size-as-a-differentiable-count.txt", "jsonld": "https://wpnews.pro/news/ngn-learning-neural-network-size-as-a-differentiable-count.jsonld"}}