Neural Networks

Researchers introduce StressGAT, a Graph Attention Network that achieves 88.62% accuracy in stress recognition by using Differential Action Units to normalize individual facial expressions relative to neutral baselines. …

Researchers introduced CvAdamW, a variant of the AdamW optimizer that monitors attention logit variance as specific heat to detect and accelerate grokking in neural networks, enabling generalization at epoch 2802 in a 40…

A new study from arXiv identifies Spectral Drift in neural network internal activations as a reliable indicator of misclassifications, with failures showing a 1.9% increase in drift (p<0.001). The researchers introduce S…

A new study tests causal emergence in an active inference agent whose architecture separates a fast perception latent z from a slow global latent g, finding that the integrated information measure Φr concentrates in g an…

Researchers propose Cardinality-Decomposed Loss (CDL) to address a silent failure in Graph Neural Networks for heterogeneous recommendation graphs, where Bayesian Personalized Ranking (BPR) causes attribute embeddings to…

Deep Q-Networks (DQN) replace the Q-table in Q-learning with a neural network that outputs Q-values for all actions, enabling generalization across similar states. To stabilize training, DQN uses experience replay to bre…

A research team hand-coded weights for one-layer MLPs that memorize labels for input token sequences of length two, finding that the number of facts memorized with 90% accuracy scales roughly linearly with parameter coun…

A developer outlines a 3-5 month plan to build a deep learning framework from scratch, culminating in a 'Tiny Transformer,' arguing that implementing an autodiff engine, tensor engine, and optimizer from the ground up pr…

PageWatch.tech has developed a computer vision diff pipeline for web change monitoring that combines SSIM, ORB feature matching, and a Siamese ResNet-18 neural network to handle layout shifts and dynamic content. The app…

Mixture of Experts (MoE) is a machine learning architecture that divides a model into specialized subnetworks called experts, with a gating network routing each input token to only the most relevant subset. This sparse a…

A developer implementing an LSTM in C/C++ for audio generation is experimenting with activation functions, including using no activation and a quadratic shaping function, and reports faster learning but slow generative o…

A guide explains that a neuron is a weighted sum followed by a non-linear activation, without which a deep network would be a linear regression. The training workflow consists of forward pass, loss calculation, backpropa…

A developer has built an upgraded version of the classic ELIZA chatbot using a sparse autoencoder with a large latent memory layer, achieving logarithmic memory scaling and resistance to catastrophic forgetting. The mode…

BrainStem, an open-source bio-inspired AI architecture using 12 digital neuromodulators for lifelong learning, has completed Stage A foundation work and legacy cleanup, with the current database containing 102,275 chunks…

Researchers introduce HyenaND, a subquadratic, global, input-dependent operator that acts directly on the native geometry of multidimensional data through convolutions with implicitly parametrized global, input-dependent…

A new unified variational framework for deep weakly supervised image segmentation uses a simplex-constrained Potts model with a smooth perimeter regularizer to create a convex, smooth energy functional that serves as a t…

A new study from arXiv (2607.19382v1) finds that explainability methods such as attention rollout and Grad-CAM can reveal how continual learning models adapt to non-stationary time series, but also expose challenges in i…

A new arXiv paper (2607.19573v1) formally defines structural generalization and proves that pure Transformers cannot learn it under the standard complexity assumption TC⁰ ≠ NC¹. The authors show that neuro-symbolic syste…

Researchers extended neural operator surrogate models for neutron flux estimation from one to two dimensions, testing Fourier neural operators (FNOs) and U-shaped neural operators (UNOs) on one-group transport with isotr…

A new study finds that orthogonalizing the mLSTM memory matrix at read time (five Newton-Schulz iterations, trained through) improves noisy associative recall by re-conditioning the learning problem during a training pla…