Researchers introduced Sigma, described as the first large-scale continuous diffusion language model at 3B and 8B parameters, built on steerable low-dimensional ODE/SDE latent trajectories and trained…
A new arXiv paper (2608.18795v1) quantifies why majority voting among LLM samples can backfire on hard questions, introducing a pluralistic agreement index Gamma decomposed into a mechanical preferenc…
Researchers propose LISA (Linear-Indexed Sparse Attention), a plug-and-play attention module that reduces inference complexity from O(n^2) to O(nM) for long-context reasoning models. Experiments on De…
Researchers propose BIRD (Bootstrapped Iterative Self-Reasoning Distillation), a two-stage method that improves compressed reasoning by first generating concise, correct traces from a base model under…
Researchers introduce KV-PRM, a process reward model that reduces scoring cost from O(L²) to O(L) by reading the KV cache from LLM generation instead of re-encoding trajectory text, achieving up to 5,…
Researchers introduced TREK (Teacher-Routed Exploration via Forward KL), a new AI training method that enhances learning through unconventional exploration strategies. TREK significantly improved perf…
A large-scale study of 53 LLM runners generating 265,000 samples found that self-consistency and cross-model agreement are weak and regime-dependent proxies for correctness, not reliable confidence si…
Toby Ord analyzes the scaling properties of reinforcement learning in AI, distinguishing between RL-scaling (training compute) and inference-scaling (deployment compute). He finds that in OpenAI's o1 …
Researchers introduced the Relative Surprisal Index (RSI), an information-theoretic metric that balances token probability and entropy to improve reinforcement learning for large language models. The …
A 3-billion-parameter model from Weibo scored 94.3 on the AIME 2026 math Olympiad, outperforming Google's Gemini 3 Pro which scored 91.7. The model is MIT-licensed and its success challenges assumptio…
Researchers introduced GRACE, a theoretical framework that determines the optimal verification granularity for test-time scaling in large language models based on problem difficulty, verifier accuracy…
Researchers have introduced On-Policy Representation Distillation (OPRD), a method that aligns student and teacher model representations across selected layers during training, bypassing the language …
Reve and Ideogram both launched new image-generation models on June 2, 2026, with each company emphasizing advances in layout control through improved labeling and code. Ideogram 4.0 is now ranked as …
OpenBMB released MiniCPM5-1B, a one-billion-parameter model that scores 40.42 on the AIME 2025 mathematics exam, outperforming larger competitors including LFM2.5-1.2B and Qwen3-0.6B. The model offers…
Researchers have developed Aryabhata 2, a reasoning-focused language model for competitive STEM examinations, trained via reinforcement learning on PhysicsWallah's internal question banks. The model o…