A small DeltaNet variant with fixed token-specific key biases and 32 KiB of recurrent matrix state reached 99.95% mean recall accuracy across three training seeds when remembering 32 new key-value pai…
A new arXiv paper (2610.00663v1) proposes Quarantined Expert Shutdown (QES), a backdoor-containment method that lets a backdoor form during training but routes trigger-conditioned behavior into a desi…
On NVIDIA Blackwell B200 GPUs, tensor-core throughput exceeds special-function exponential throughput by more than two orders of magnitude, exposing exponential evaluation as a bottleneck in fused att…
A mechanism-aware conditioning framework that injects ensemble-covariance statistics into an otherwise unchanged backbone improves rare-event emulation in data-limited chaotic systems, according to ar…
Zalando tested a daily machine-learning pricing system across 23 A/B tests in 12 European markets and found it improved profit during sales without reducing revenue or units sold after returns, accord…
A developer argues that the next leap in LLM capability will come from human-like memory mechanisms rather than scaling laws alone, contending that finite attention windows cannot control ever-rising …
A new arXiv paper (2609.27233v1) introduces a method for distilling sequential computation in Transformer language models by replacing spans of input tokens with collapsed surrogate embeddings generat…
Transformer inference requires a KV cache to cut generation complexity from O(n^2) on the feedforward network and O(n^3) on attention down to O(n) and O(n^2) respectively, according to Part 7 of the "…
A blog post titled "Lean Verified Transformers" presents a Lean formalization of foundational Transformer properties — including tensor parallelism, data parallelism, batch invariance, permutation inv…
The Transformer neural network architecture, introduced in the 2017 paper "Attention is All You Need," has become the go-to deep learning architecture for text-generative models including OpenAI's GPT…
Researchers released FairLMs, a Python library that unifies fairness measurement and mitigation for language models through explicit declarations of model capabilities and input requirements, accordin…
A comparative study of temporal deep learning architectures for physiological emotion recognition found the Transformer achieved the highest multimodal accuracy on the WESAD dataset at 99.02% +/- 0.51…
Researchers introduced the Knowledge-enhanced Agriculture-informed Neural Network (KAINN), a hybrid neural-mechanistic framework that extends the Agriculture-informed Neural Network (AINN) by incorpor…
A 10,000-API-call probe of TypeSafe's Jev API concludes the system is likely a causal transformer, possibly using sparse mixture-of-experts, repurposed to read decision probabilities directly from int…
A new arXiv paper (2609.17571v1) reports that grokking in Transformers is a spectral recoding of an existing distributed circuit rather than a switch between modules, based on behavior-aligned exact a…
A new arXiv paper (2609.16752v1) introduces the Cognitive Field Network (CFN), a recurrent Transformer in which an organized hidden field re-enters subsequent inference via the update rule Phi_{n+1}=F…
A developer has published a solution to a puzzle from a workshop on building a minimal hand-constructed transformer, demonstrating why position embeddings and residual connections are necessary. The e…
Researchers introduced Grouped Value Attention, a method that reduces the Transformer KV cache by reconstructing keys on demand rather than storing them at every decoding step. The technique builds on…
A study posted to arXiv proposes a hybrid Temporal Convolutional Network (TCN)-Transformer model to predict the probability of collision (PoC) expected in the next Conjunction Data Message (CDM) updat…
Offline reinforcement learning post-training can substantially improve zero-shot code generation in large language models without online sampling, according to an arXiv paper (arXiv:2609.11956v1). The…