Neural Networks

A new arXiv paper (2607.13042v1) traces how PyTorch's automatic differentiation engine computes gradients for Physics-Informed Neural Network training, using a 1-3-3-1 multilayer perceptron and the initial value problem …

Researchers at MPI Informatics have developed a differentiable polarized path tracing method that enables stable gradient estimation for inverse rendering, overcoming numerical instability caused by rank-deficient polari…

Researchers propose Samba, a hybrid Mamba architecture for audio-visual navigation that replaces conventional GRUs with an adaptive selection-enabled Mamba State Encoder (M-SE) and introduces an Audio Mamba Encoder (AME)…

Researchers propose targeted parameter decomposition (tPD), a method that extracts only the neural network components responsible for specific inputs, reducing compute costs. In tests on transformer language models train…

Researchers propose Differentiable Geometry Image (DiffGI), an end-to-end 3D-to-2D mapping framework that replaces binary occupancy maps with a continuous 2D Truncated Signed Distance Function (TSDF) to eliminate stairca…

A new arXiv paper (2607.13073v1) expands the cognitive power of neuro-symbolic AI based on IFOL_B by integrating probability computation for unknown sentences using Nilsson's probability structure. The authors introduce …

A 200-step, ~1.7B-parameter ablation run in OLMo-core comparing Engram on a standard attention Transformer versus a 3 GDN-layer + 1 attention-layer hybrid found that Transformer + Engram reached slightly lower cross-entr…

Researchers have released SynapticOS, an open-source inference-first runtime for neural processing units on resource-constrained microcontrollers, built on Zephyr. The runtime achieves constant-time memory allocation (~1…

Chinese neurotechnology firm Neuracle (Borui Kang Medical Technology) has performed the world's first successful surgical implant of a commercial brain-computer interface, a coin-sized device called Neural Electronic Opp…

A new mechanistic interpretability study of the MAIA 3 chess transformer finds that knight-fork detection is primarily assembled compositionally from check and queen-attack subcomponents, with causal ablation evidence ty…

A diffusion model's creativity is a mathematical consequence of neural networks learning a 'smoothed' version of the score function, driving the model to interpolate between training data points along the hidden data man…

Researchers at an undisclosed institution trained a neural network that performs computation in superposition (CiS), implementing more nonlinear functions than it has neurons, by switching from L² to L⁴ loss and removing…

A new benchmark, GLNBench, reveals that graph neural networks (GNNs) collapse under label noise when using strong backbones, with geometry-based methods like ERASE and UnionNET suffering rank collapse to near one on 512-…

Geoffrey Hinton, the British-Canadian computer scientist whose foundational work on neural networks and deep learning underpins modern artificial intelligence, won the 2024 Nobel Prize in Physics jointly with John Hopfie…

FFAvatar introduces a Transformer-based 3D Gaussian framework for constructing animatable 4D head avatars from a few reference images, with quality improving as more images are added. The system uses an alternating atten…

The Neuro-Symbolic Knowledge- and Gradient-Guided Reinforcement Learning (KGRL) algorithm, developed to exploit domain knowledge stored in a Datalog knowledge base, significantly outperforms state-of-the-art RL baselines…

Researchers have developed a multi-feature fusion framework that combines static lexical representations (Word2Vec) with dynamic contextual representations (GPT) via a non-linear cross-attention mechanism, achieving supe…

A new developer published an introductory article on neural networks aimed at beginners, explaining core concepts without requiring a mathematical background.

A compact Karpathy-style decoder experiment comparing real, complex, and quaternion Transformer projections found that quaternion projections achieve lower validation loss through 2 million training tokens, but real proj…

Researchers have introduced LOD-MSNO (LOD-Multiscale Neural Operator), a hybrid model that combines the Localized Orthogonal Decomposition method with neural operators to improve accuracy on multiscale problems while mai…