{"slug": "correction-training-for-diffusion-models", "title": "Correction Training for Diffusion Models", "summary": "A new arXiv paper (2610.14136v2) introduces a correction step for diffusion models that reduces the training-inference gap caused by exposure bias, where the denoiser is trained on real images interpolated with noise but at inference receives its own imperfect predictions. The corrector takes the base model's clean-sample prediction as input, is trained against the same ground-truth target, and is applied iteratively at each denoising step; evaluated across DiT, SiT, and JiT, it substantially improves FID scores for unguided and unconditional generation, achieves performance competitive with classifier-free guidance (CFG), and combining correction with CFG further improves diversity and reduces the oversaturation typical of CFG. The authors also show the additional inference cost can be amortized by distilling the refined predictions back into the base diffusion model, retaining quality gains without extra inference-time overhead.", "body_md": "arXiv:2610.14136v2 Announce Type: new \nAbstract: Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noise, but at inference its inputs are constructed from its own imperfect predictions from the earlier steps, something it has never seen at training time. This might be one reason diffusion models often struggle without additional help, such as CFG. We introduce a correction step for diffusion models which attempts to reduce this training-inference gap. The corrector takes the base model's prediction as input and is trained against the same ground-truth target. This makes correction an explicit learning task, with training inputs provided by the base model. During sampling, at each denoising step, we iteratively refine the denoiser's clean-sample prediction. This allows us to correct predictions at intermediate denoising steps, preventing early artifacts from persisting into the final sample. Although the corrector is trained only on predictions from the base denoiser, it can be applied repeatedly to its own outputs at inference for better results. This simple framework leads to substantial improvement in sample quality while largely preserving diversity. We evaluate our method across DiT, SiT, and JiT. Our method substantially improves upon previously reported FID scores for unguided generation and, notably, unconditional generation, where producing high-quality samples without guidance has been particularly challenging. Our method achieves performance competitive with classifier-free guidance (CFG), and combining correction with CFG further improves diversity and reduces oversaturation typical of CFG. Finally, we show that the additional inference cost of correction can be amortized by distilling the refined predictions back into the base diffusion model, retaining the quality gains without additional inference-time overhead.", "url": "https://wpnews.pro/news/correction-training-for-diffusion-models", "canonical_source": "https://arxiv.org/abs/2610.14136", "published_at": "2026-10-15 04:00:00+00:00", "updated_at": "2026-10-15 05:43:46.812495+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "generative-ai", "ai-research", "neural-networks"], "entities": ["arXiv", "DiT", "SiT", "JiT", "classifier-free guidance"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/correction-training-for-diffusion-models", "markdown": "https://wpnews.pro/news/correction-training-for-diffusion-models.md", "text": "https://wpnews.pro/news/correction-training-for-diffusion-models.txt", "jsonld": "https://wpnews.pro/news/correction-training-for-diffusion-models.jsonld"}}