cd /news/large-language-models/see-every-block-once-variance-reduce… · home › topics › large-language-models › article
[ARTICLE · art-170339] src=machinebrief.com ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

See Every Block Once: Variance-Reduced Post-Training for Masked Diffusion Language Models

A full-block estimator that evaluates every block exactly once and averages the results improves post-training for Masked Diffusion Language Models, according to arXiv paper 2611.17754v1. The authors prove the estimator stays unbiased while eliminating between-group variance from block selection, yielding lower variance than the random block sampling baseline at the same sample size. Across five benchmarks spanning mathematics, planning and coding, the method posted the highest accuracy on four and beat the random block sampling baseline by an average of 12.79 points.

by read1 min views1 publishedNov 16, 2026

arXiv:2611.17754v1 Announce Type: new Abstract: Policy gradient methods are widely used to align large language model outputs with human preferences, but the log-likelihoods they require cannot be directly computed in Masked Diffusion Language Models (MDLMs). MDLM post-training therefore relies on a surrogate objective based on the Evidence Lower Bound (ELBO), and recent work estimates a blockwise variant of this objective via Monte Carlo methods to support blockwise generation. However, existing estimation schemes randomly sample the block index, so they may evaluate the same block more than once and leave some blocks unevaluated. This introduces variance arising from block selection into the estimates. In this work, we propose a full-block estimator that evaluates every block exactly once and averages the results. We theoretically show that the proposed estimator remains unbiased while eliminating the between-group variance arising from block selection, thereby achieving lower variance than the baseline for the same sample size. In experiments on five benchmarks spanning mathematics, planning, and coding, our method achieves the highest accuracy on four benchmarks and outperforms the random block sampling baseline by an average of 12.79 points.

── more in #large-language-models 4 stories · sorted by recency
── more on @masked diffusion language models 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/see-every-block-once…] indexed:0 read:1min 2026-11-16 · —