The Transformer architecture, introduced in the 2017 paper 'Attention Is All You Need', revolutionized AI by replacing sequential RNNs with a parallelizable attention mechanism. This mechanism allows …
Researchers introduced Ghost Attractor Networks, a dynamical decoder that uses basin-structured latent representations for closed-loop sequential generation. The 2.3-million-parameter Ghost matched th…
Researchers introduced Gaussian Mixture Attention (GMA), a probabilistic attention mechanism that replaces pairwise query-key comparisons with routing through K learned Gaussian mixture components, ac…
Noam Shazeer, co-inventor of the transformer architecture and co-lead of Google's Gemini program, is leaving Google to join OpenAI, despite Google paying approximately $2.7 billion to retain him less …
A developer implemented a full Transformer with KV cache on an FPGA, achieving over 56,000 tokens per second at only 80 MHz, without using a GPU or CPU. The design was created gate by gate as a custom…
Researchers achieved 98.91% accuracy in multimodal emotion recognition from physiological signals by combining LSTM, TCN, and Transformer models with late-fusion ensemble on the WESAD dataset. Transfo…
Researchers introduced a hierarchical model for ball action anticipation in football broadcast video, achieving 17.91% mAP on the SoccerNet benchmark. The system uses a GRU and Transformer decoder wit…
A developer argues that balanced ternary (-1, 0, +1) could replace binary for AI hardware, citing 20× model compression, 3× inference speedup, and 8× power reduction. Microsoft's BitNet b1.58 demonstr…
Recursive's automated AI research system achieved state-of-the-art results on three benchmarks: fixed-budget language model training, small-model training speed, and GPU kernel optimization. The syste…
A developer explains how the Transformer architecture works, from self-attention to modern LLMs. The key innovation is that Transformers compare tokens directly via attention rather than processing se…
Researchers developed a Transformer-based scheduling policy for the open shop scheduling problem (OSSP) using deep reinforcement learning. The model, trained on small Taillard benchmark instances, gen…
A developer argues that the current AI revolution is fundamentally different from past waves of enthusiasm, citing the convergence of large-scale labeled data, GPU computing, and deep network architec…
Jürgen Schmidhuber, a pioneer in artificial intelligence whose lab developed foundational ideas like LSTM, world models, and artificial curiosity decades before they became mainstream, argued in a new…
NVIDIA released Nemotron 3 Ultra, a 550-billion-parameter open Mixture-of-Experts model designed for long-running AI agents that plan, use tools, and reason across many turns. The model uses a hybrid …
A developer has demonstrated that the 1982 Hopfield associative memory update rule and the 2017 Transformer scaled dot-product attention mechanism are mathematically identical operations, with one equ…
Michigan Senate candidate Mallory McMorrow unveiled a detailed artificial intelligence policy agenda last week, proposing an AI Workforce Reinvestment Fund and a "token tax" on companies using AI to f…
A systematic study of query, key, and value (QKV) projection variants in Transformers found that sharing the key and value projections (Q-K=V) performs on par with or better than the standard three-pr…
Five foundational research papers explain how large language models work, covering the Transformer architecture, in-context learning, scaling laws, and instruction tuning with human feedback. The pape…
Researchers have developed TopoMamSurv, a Graph Mamba survival analysis framework that uses topology-aware ordering to address computational bottlenecks in Whole Slide Image analysis. The framework in…
Researchers introduced Geometry-Aware Tabular Diffusion (GATD), a method that improves tabular data synthesis by feeding pairwise angles and lengths from column value differences into diffusion denois…