Neural Networks

Keith Thomas, paralyzed from the chest down after a swimming accident six years ago, has regained movement in his arms and hands through a brain implant known as a 'double neural bypass,' allowing him to feed himself for…

Neurologists at the Feinstein Institutes for Medical Research in New York reported in a study published today in Nature Medicine that a man with quadriplegia, Keith Thomas, can now feed himself and drink from a cup using…

A new universal operator approximation theorem for encoder-decoder architectures, developed by researchers, challenges traditional constraints by operating independently of compact sets and accommodating normed and metri…

A new modular Context-Augmented Prompting framework boosts small language model accuracy in molecular property predictions by up to 74% on the Tox21 dataset, using graph-derived context from a trained Graph Neural Networ…

MxGPS, a novel multiplex graph transformer, achieves a 0% boundary violation rate across four unseen power grid topologies, demonstrating that topology overfitting—not model capacity—is the primary barrier to generalizat…

Skip connections and normalization layers in Transformer architectures serve a critical function beyond gradient stabilization: they preserve the rank of representations across layers, preventing rank collapse that would…

Neural networks trained on modular arithmetic tasks either succeed immediately or fail completely, according to research showing that algebraic constraints limit their expressible function class. Across 585 simulation ru…

FastCentNN, an upgrade to Centroid Neural Networks, reduces runtime by up to 16% on synthetic 2D datasets and about 5% on high-dimensional ones while maintaining clustering quality, according to its developers. The algor…

Researchers at the University of Manchester are advancing neuromorphic computing, a brain-inspired approach that could make AI and sensing far more energy-efficient, with projects including the world's largest neuromorph…

Researchers have discovered a direct link between the eigenvalues of the Hessian matrix and the sharpness of classification solutions in deep neural networks, with sharper solutions correlating with better generalization…

A novel convolution-inspired network structure for classifying ancient cuneiform tablets outperforms the state-of-the-art transformer-based network Point-BERT, according to researchers who developed the architecture. The…

Large language models like OLMo-2 7B exhibit left-right brain asymmetry in language processing that mirrors the human brain's left-hemisphere dominance, according to research using fMRI. As the models train, their intern…

A new AI architecture called Dendritron Transformer, designed to never forget, is explained in a video. The system aims to address catastrophic forgetting in neural networks by incorporating dendritic computation. This a…

The encoder-decoder (seq2seq) architecture, proposed by Cho et al. and Sutskever et al. in 2014, compresses the entire source sentence into a single fixed-length vector, causing performance to degrade sharply on longer s…

Recurrent neural networks (RNNs) process sequences by maintaining a hidden state that compresses the entire history into a fixed-size vector, as formalized in Elman's 1990 paper "Finding Structure in Time." The hidden st…

Google researchers led by Tomas Mikolov published word2vec in 2013, a method that made high-quality dense word embeddings cheap to train at billion-token scale on a single machine, transforming them from a research curio…

Gradient descent is the core algorithm behind deep learning, enabling neural networks with millions of parameters to minimize loss by iteratively adjusting weights in the direction opposite to the gradient. The gradient,…

A blog post explains that the L2 norm measures a vector's magnitude as its Euclidean distance from the origin, while the dot product quantifies similarity between vectors and is the core operation behind attention mechan…

The first post in the 'Holding the LLM Stack in Your Head' series rebuilds intuition for linear algebra, explaining that vectors and matrices are the foundation of neural network operations. The author defines a 'space' …

Researchers propose Differentiable Geometry Image (DiffGI), an end-to-end 3D-to-2D mapping framework that replaces binary occupancy maps with a continuous 2D Truncated Signed Distance Function (TSDF) to eliminate stairca…