Neural Networks news and analysis on Web Pulse: 1261 curated articles tracking the latest Neural Networks developments, tools, and research, updated continuously from vetted sources.
A new companion app visualizes the internal workings of a chess transformer model that mimics human play, allowing users to explore attention heads and residual stream evolution. The tool, linked from a GitHub repository…
A developer trained a Joint-Embedding Predictive Architecture (JEPA) world model, called LeMario, on Super Mario Bros from scratch, finding the model could predict five-step futures and perform reward-free planning for n…
GPT is a generative pre-trained transformer, a neural language model that processes token sequences to predict the next token. Understanding GPT requires examining tokenization, Transformer architecture, pre-training, pa…
Researchers have introduced Multi-Scale Convolution with Optimal Transport Attention (MSC-OT), a novel architecture that combines multi-scale convolution with Sinkhorn optimal transport to enhance attention mechanisms fo…
A novel framework decouples feature transformation from network topology in Graph Neural Networks (GNNs) to enhance structural robustness, introducing the Channel-Split Adaptive Gated GNN (CSAG-GNN) with a node-wise gati…
Researchers have introduced Weight-Adjusted Gradients (WAG), a novel method for estimating parameter importance in Large Language Models that reveals a tiny subset of parameters disproportionately influencing model behav…
Researchers have discovered that deep neural models for click-through rate predictions suffer a sharp decline in validation performance after just one training epoch, even as training loss improves, due to overfitting. B…
A new neural network optimization method using a symmetric-exponential weight reparameterization achieved up to 30% faster training on transformers, reaching equivalent validation loss in 1.32 to 1.49 times fewer steps o…
A Redwood Research project found that covert communication between AI agents can evade detection through geometric movement rather than obfuscation. In experiments using a 216M-parameter SpikeGPT neural network, a hidden…
Researchers have introduced a novel approach using normalizing flows, specifically RealNVP, to model human body poses in 6D rotation format, offering a new probabilistic foundation for motion capture. The method inverts …
Researchers have developed a deep-learning method for MR elastography that reconstructs high-resolution images from highly undersampled data, achieving 2mm isotropic resolution in one minute with a total R value of 10. T…
A neural network-based model developed by researchers replicates how infants learn to control their movements in the mobile paradigm, revealing that spontaneous movements after disconnection are not random but part of se…
Researchers have introduced GradSkip, a new method for Vision Transformers that uses adaptive head weighting and re-evaluates residual connections to achieve a 14-times reduction in GFLOPs while improving faithfulness on…
Researchers have demonstrated that forcing neural networks to adopt low-dimensional representations significantly enhances their ability to generalize, according to a new study. Using information-theoretic measures, the …
New research shows depth is more important than width in neural networks for approximating analytic functions, challenging traditional thinking. The study derived new approximation rates expressed as O(N^{-CL^τ}), where …
The Multi-scale Feature Enhanced Graph Neural Network (ME-GNN) achieved a relative L2 error of 0.0196 for velocity fields on the ShapeNet-Car benchmark and a normalized mean squared error of 0.0033 for flow fields on Air…
SPARC-Net, a novel neural network architecture, claims to overcome the limitations of Physics-Informed Neural Networks (PINNs) in solving stiff and shock-dominated partial differential equations, achieving error reductio…
A new unified multi-dimensional priority-constraint framework for physics-informed neural networks (PINNs) prioritizes training regions based on natural physical information flow, using negative-exponential residual weig…
Landmark-based embeddings on inhomogeneous random graphs offer tighter dimension-distortion trade-offs than traditional worst-case bounds, improving distance preservation in graph machine learning. A GNN-augmented varian…
Multihead attention in transformer models uses 12 layers each with 12 attention heads, totaling 144 heads, to read a sentence from different angles. Each head produces a length-64 update that is concatenated and recombin…