Stanford researchers introduced TRACE, an open-source system that diagnoses recurring agent failures and trains targeted capabilities using synthetic reinforcement learning environments. TRACE identif…
Researchers introduced TriRoute, a unified controller that jointly optimizes attention resolution, expert selection, and KV-cache bit-width for language models, achieving Pareto-dominant performance o…
Google DeepMind released Gemma 4, a new generation of open-weight multimodal language models featuring dense and Mixture-of-Experts architectures from 2.3B to 31B parameters. The models introduce a th…
NVIDIA released NeMo AutoModel, an open-source library that accelerates fine-tuning of Mixture-of-Experts (MoE) transformer models by 3.4-3.7x in training throughput and reduces GPU memory usage by 29…
MiniMax released MSA (MiniMax Sparse Attention), a block-sparse attention method that reduces the quadratic cost of softmax attention at long contexts. The method was trained on a 109B-parameter Mixtu…
Researchers found that repetition loops in Gemma 4 instruction-tuned LLMs can be fixed by editing as few as one neuron, but the fix does not resolve deeper 'doom loops' caused by missing knowledge. Th…
Mixture-of-Experts (MoE) architecture allows large language models to use only a fraction of their total parameters for each token, enabling models with 671 billion total parameters to run at speeds c…
JetBrains released Mellum2, a 12-billion-parameter Mixture-of-Experts model with 2.5 billion active parameters per token, under the Apache 2.0 license. The model is specialized for software engineerin…
StepFun released Step 3.7 Flash, a 198-billion parameter Mixture-of-Experts vision-language model with native image understanding and improved tool-use reliability for agentic coding and search workfl…