Neural Networks

A researcher reports that a byte transformer with a zero-parameter input layer (HSL-embedding-zero) performs comparably to learned embeddings across text, image, audio, radar, and lidar modalities, with channel-knockout …

INFN-Padova physicist Tommaso Dorigo announced a two-year research position tied to the EU PHINDER project, targeting Master-level graduates with machine learning experience. The application deadline of May 30, 2026 has …

Scientists have found that genetic risk scores for ADHD directly predict irregular timing in brain waves responsible for focus and goal-directed behavior. The study, published in Translational Psychiatry, links polygenic…

A blog post on fuzzy sets, fuzzy logic, and fuzzy inference was submitted to Hacker News, receiving 1 point and no comments.

A developer implemented the SDAR gate, a gated distillation mechanism for reinforcement learning with language models, in PyTorch. The gate uses a sigmoid function to weight a per-token KL divergence loss based on the co…

A new AI research video explores how world models enable artificial intelligence to develop a first-person perspective, potentially advancing autonomous decision-making and machine consciousness studies.

Researchers developed custom Triton kernels that accelerate JumpReLU Sparse Autoencoder inference by 2–14× on real SAEs, exploiting activation sparsity to skip zero entries during matrix multiplication. The kernels use e…

Rudrite Research published a comparison of three post-Transformer sequence models — S4, Mamba, and RWKV — highlighting their approaches to achieving linear computational cost while maintaining quality. The analysis expla…

Rudrite Research published a comparison of three major transformer-based language models—BERT, GPT, and T5—detailing their distinct pretraining approaches: bidirectional encoding, autoregressive next-token prediction, an…

A developer built Frank Rosenblatt's 1958 Perceptron from scratch, implementing a single neuron with weighted inputs, bias, and step activation. The project demonstrates how a perceptron learns a linear decision boundary…

A developer launched Traindeep.ai, a web application that generates trained neural network models for microcontrollers such as Raspberry Pi Pico and ESP32. The tool allows users to load a CSV file, dynamically maps input…

Researchers Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, and Shiwei Liu published a paper on arXiv titled 'The Curse of Depth in Large Language Models,' investigating performance degradation in deep LLM…

A developer completed Week 1 of Course 2 (Advanced Learning Algorithms) in the Machine Learning Specialization, covering neural networks intuition, models, forward propagation, and TensorFlow implementation. The develope…

Scientists at the Icahn School of Medicine at Mount Sinai mapped the brain's energy landscape in people with major depressive disorder, finding that altered white matter pathways trap the brain in maladaptive activity st…

A developer outlines a strategy for on-device training of lightweight neural networks in iOS apps, enabling personalized predictions while preserving user privacy. The approach uses a compact MLP trained locally on user …

Larkos 0.3 introduces a GAT-based neuron graph reasoner, a temporal attention encoder, and a refactored fusion head. The update replaces mean-pooling with learned-query attention and increases memory capacity, while the …

A new coding implementation demonstrates how to infer urban functions using spatial graph neural networks. The pipeline collects POI and street network data from OpenStreetMap, constructs proximity graphs, and trains a G…

A researcher applied activation patching to a DNA sequencing basecaller, a deep learning model, to understand its internal mechanisms, finding MLP dominance in early and late layers and concentrated self-attention activi…

A developer explains the sparse Mixture of Experts (MoE) architecture used in models like Mixtral, DeepSeek-MoE, and Grok-1, detailing how the router selects which experts to activate per token and why load-balancing is …

A Nature preprint published June 13, 2026, introduces CURE, a graph unlearning framework that uses contrastive representation editing to remove the influence of target nodes, edges, or features from a trained graph neura…