Neural Networks

A developer explains the importance of positional embeddings in Transformers, comparing Absolute Positional Embedding (APE), Relative Positional Embedding (RPE), and Rotary Positional Embedding (RoPE). APE adds a positio…

A developer released OmniCore, a lightweight neural network engine that serves as an external behavioral brain for game NPCs, enabling multimodal agency through environmental perception, dual-control navigation, and modu…

A developer proposes an experimental method for storing neural network weights using a tree structure that stores unique values and allows weights to reference them by index, potentially reducing memory usage when many w…

Researchers introduced Mapping Networks, a novel approach that replaces high-dimensional weight spaces with compact latent vectors, achieving a 500x reduction in trainable parameters while maintaining comparable or bette…

A developer introduced OmniCore, a neural engine for multimodal NPC agency currently in beta. The engine aims to provide a neural brain for game NPCs, enabling more dynamic and intelligent behavior.

Researchers from Microsoft Research, UC Berkeley, UC San Francisco, and Columbia University developed generative causal testing (GCT), a method that distills black-box brain-prediction models into testable verbal explana…

Neural network weights are initialized by dividing by the square root of the number of inputs to prevent the network from being dead before training begins. This technique, known as Xavier initialization, ensures stable …

Researchers from the University of Cambridge and the University of Oxford have derived conditions under which data augmentation yields exact equivariance in Bayesian neural networks trained with variational inference. Th…

Researchers developed a multi-task spatiotemporal deep neural network that predicts penetration depth and weld seam morphology in laser welding with high accuracy, achieving 99.35% accuracy for penetration state and 95.6…

Researchers developed an attention-based, physics-guided convolutional neural network to predict microstructural evolution in binary mixtures governed by the Cahn-Hilliard equation. The surrogate model accurately forecas…

A new arXiv paper reviews neural architecture search (NAS) methods for generative adversarial networks (GANs), finding that evolutionary algorithms and gradient-based methods outperform others in optimizing GAN design. T…

Researchers developed topology-informed neural networks that integrate topological data analysis with deep learning to improve flood detection in satellite imagery, achieving more robust and interpretable results on the …

Researchers extended fMRI cognitive taskonomy from single-source to multi-source transfer across 23 Human Connectome Project task states, training 1,127 models. They found that motor states transfer well within their par…

Researchers propose GeMoE, a dynamic routing method for Mixture-of-Experts-based large vision-language models that uses gating entropy to adaptively select the number of experts per token, achieving 99.5% performance ret…

Researchers introduced Neural Voxel Dynamics, a self-supervised framework that learns implicit 3D physics from video by lifting 2D features into a volumetric latent space. The method achieves long-term structural stabili…

Researchers propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action-prefix prediction, improving average success over LeWM while substantially reducing planning t…

Scaling laws in deep learning describe a predictable power-law relationship between training loss and model size, dataset size, and compute, enabling extrapolation of resource requirements for larger models. Early theore…

An automotive engineer faces 26-hour design iterations for CFD simulations of a side mirror, motivating a new generation of AI tools like PhysIQ that aim to accelerate physics simulation. This primer explains the basics …

The softmax function, essential to modern AI systems like GPT, originated from Ludwig Boltzmann's 19th-century work on thermodynamics, where he derived the exponential form to maximize entropy in gas molecules. The same …

A machine learning researcher pitted 21 algorithms against each other in a regression task using a 512x512 image as the target function, but artificially inflated the input space to 500, 1000, and 2000 features, with onl…