{"slug": "data-free-multistep-diffusion-distillation", "title": "Data-free Multistep Diffusion Distillation", "summary": "Researchers proposed Progressive Multistep Data-free Distillation (PMDD), a data-free multistep diffusion distillation method that assigns a dedicated adapted network per student distribution to avoid the biased distribution matching caused by simultaneous training. PMDD reaches FID 1.99 on ImageNet 64x64 in two steps, below its 512-NFE teacher's 2.32, and achieves FID 7.76 on zero-shot COCO 512x512, while using 4-5x less training compute than data-free DMDv2 to support three inference budgets. The work, posted as arXiv:2610.19305v1, adds a stage-wise unforget loss to preserve earlier-step generations and yields a single unified model supporting multiple inference budgets.", "body_md": "arXiv:2610.19305v1 Announce Type: new \nAbstract: One-step diffusion distillation achieves strong performance on small datasets but remains insufficient for high-resolution or complex text-to-image settings. Extending to multistep distillation often requires training data, suffers from early-step degradation, inconsistent outputs across sampling budgets, and incurs high computational cost. We identify that these limitations stem from simultaneous training, where a single adapted network approximates multiple score functions of different student distributions at once, causing biased distribution matching and unstable optimization. Therefore, we propose Progressive Multistep Data-free Distillation (PMDD), which assigns a dedicated adapted network per student distribution while avoiding the memory overhead of maintaining multiple adapted networks. Combined with a stage-wise unforget loss to preserve earlier-step generations, PMDD reduces optimization bias in the student, provides faster convergence, and a single unified model that supports multiple inference budgets. Extensive experiments show that PMDD matches or outperforms data-free few-step baselines at every multistep budget, reaches FID 1.99 on ImageNet 64x64 in two steps, below its 512-NFE teacher (2.32), and achieves FID 7.76 on zero-shot COCO 512x512, while requiring 4-5x less training compute than data-free DMDv2 to support three inference budgets.", "url": "https://wpnews.pro/news/data-free-multistep-diffusion-distillation", "canonical_source": "https://arxiv.org/abs/2610.19305", "published_at": "2026-10-15 04:00:00+00:00", "updated_at": "2026-10-15 05:56:17.453425+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "generative-ai", "computer-vision", "ai-research"], "entities": ["PMDD", "Progressive Multistep Data-free Distillation", "ImageNet", "COCO", "DMDv2", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/data-free-multistep-diffusion-distillation", "markdown": "https://wpnews.pro/news/data-free-multistep-diffusion-distillation.md", "text": "https://wpnews.pro/news/data-free-multistep-diffusion-distillation.txt", "jsonld": "https://wpnews.pro/news/data-free-multistep-diffusion-distillation.jsonld"}}