Serving an open-weight large language model in a user-facing product is primarily a latency problem, and open weights enable engineers to control inference speed through levers such as weight quantiza…
A new arXiv study (2609.04482v1) introduces a boundary-aware self-distillation framework for controlled LLM safety refusal, showing that escalating retries reduce prompts without accepted refusal trac…
A new arXiv preprint (2609.04518v1) finds that in multi-harness reinforcement learning for coding agents, the evaluation harness is the dominant variable, moving the mean solve rate from 2.14% to 9.27…
Researchers propose Gradient-Aligned Reward (GAR), a dense reward method for reinforcement learning from verifiable rewards (RLVR) that uses cosine similarity between rollout and expert-anchor gradien…
Researchers introduced AdaptiveSpec, a training-free per-step speculative decoding method that adapts both draft verification and tree shape from internal decoding signals, improving throughput over E…
Researchers propose CARE (Contrastive Anchor-based Rubric Evolution), a method that uses anchor responses from frontier models to dynamically evolve rubrics during LLM post-training, addressing reward…
DeepSeek's DSpark speculative decoding technique, which combines parallel drafting with a lightweight sequential component, can improve local LLM generation speed on the same GPU, with DeepSeek report…
Researchers from Meta AI and the University of Illinois Urbana-Champaign introduced EvoHarness-RL, a trainable coordination layer that enables large language model agents to create and manage their ow…
XPress, a lightweight causal refiner for diffusion drafters in speculative decoding, raises acceptance length by ~30% on average (up to +56%) and decoding throughput by ~1.3× on average (up to 1.7×) c…
Researchers introduced Tensor Mixture (MixT), a general tensor-structured compression scheme that replaces dense linear layers in large language models with natively executable mixtures of tensor oper…
Researchers introduced Influence-Aware Policy Optimization (IAPO), a method that uses typed influence-dependency graphs to redistribute trajectory-level advantages for credit assignment in multi-turn …
An engineer at Jahn AI ran a per-prompt regression check before recommending FP8 quantization for Qwen3-8B serving on an RTX PRO 6000 Blackwell, which delivered a 1.5x throughput gain (1,725 to 2,597 …
An engineer benchmarked Qwen3-8B on workstation Blackwell GPUs across three serving stacks, finding that vLLM outperformed SGLang and llama.cpp by up to 4x under load, and that an FP8 pass delivered a…
A new arXiv study (2608.20362v1) finds that exact-match verifiers in reinforcement learning with verifiable rewards (RLVR) introduce language-dependent false-negative reward noise in multilingual sett…
A new arXiv study (2608.20344v1) finds that the main constraint in LLM-based digital twins is not information volume but how persona information is structured, with a hand-crafted schema (BDE: Backgro…
Alibaba Group Holding Ltd. increased its capital expenditure by 75% to build AI infrastructure despite a 76% drop in quarterly net income, while releasing open-source Qwen3-8B and Qwen3-27B models tha…
A data scientist at an unnamed company cut AI API costs by 95% by analyzing six months of logs and implementing a model-routing pipeline that matches each request to the cheapest adequate model. The a…
A bootcamp graduate's deep dive into the cheapest AI APIs of 2026 reveals that ultra-budget models like Qwen3-8B and GLM-4-9B cost as little as $0.01 per million output tokens, while the sweet spot ti…
A new method called Forward-Pass-Only MLP training (FPO) adapts large language models without a backward pass, achieving 2.7–3.2x the throughput of standard fine-tuning at ~40% less peak training memo…
MiniMax released MiniMax-Music3, an open-weights text-to-music model that generates complete songs of up to five minutes from lyrics and a structured caption, outputting 32 kHz, 16-bit stereo WAV. The…