{"slug": "dualheads-reward-tilted-few-step-diffusion-made-simple", "title": "DualHeads: Reward-tilted few-step diffusion made simple", "summary": "A new arXiv paper (2610.28858v1) introduces DualHeads, a method that adapts few-step distilled diffusion models to task and human preferences using only external reward evaluations and reference samples, implemented as two lightweight binary classifier heads on a frozen diffusion backbone. The reward head learns which samples of the same prompt score higher and the reference head separates reference images from generated ones, with the generator maximizing a weighted sum of their logits; when the heads are optimal, the objective is KL-regularized maximization of the learned reward, with the ratio of head weights setting regularization strength. With both rule-based and model-based rewards, a few-step DualHeads generator reaches task rewards close to those of 40-step reinforcement learning methods while preserving image quality at no extra inference cost.", "body_md": "arXiv:2610.28858v1 Announce Type: new \nAbstract: Distilled diffusion models generate high-quality images in a few sampling steps, but aligning them with task requirements and human preferences is difficult. Methods developed for multi-step diffusion models rely on differentiable rewards, or on score fields and stochastic trajectories that distilled generators may lack. We ask whether a few-step generator can be adapted using only external reward evaluations and reference samples, and show that two binary classifiers suffice. Our method, DualHeads, implements them as lightweight heads on a frozen diffusion backbone. The reward head learns which samples of the same prompt score higher, the reference head separates reference images from generated ones, and the generator maximizes a weighted sum of their logits. When the heads are optimal, this objective is KL-regularized maximization of the learned reward, and the ratio of head weights sets the regularization strength. Empirically, with both rule-based and model-based rewards, a few-step DualHeads generator reaches task rewards close to those of 40-step reinforcement learning methods while preserving image quality, at no extra inference cost. We hope that this simple and elegant dual-head formulation will help advance few-step reward-tilted generation.", "url": "https://wpnews.pro/news/dualheads-reward-tilted-few-step-diffusion-made-simple", "canonical_source": "https://arxiv.org/abs/2610.28858", "published_at": "2026-10-22 04:00:00+00:00", "updated_at": "2026-10-22 04:11:21.493300+00:00", "lang": "en", "topics": ["generative-ai", "machine-learning", "ai-research", "neural-networks"], "entities": ["DualHeads", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/dualheads-reward-tilted-few-step-diffusion-made-simple", "markdown": "https://wpnews.pro/news/dualheads-reward-tilted-few-step-diffusion-made-simple.md", "text": "https://wpnews.pro/news/dualheads-reward-tilted-few-step-diffusion-made-simple.txt", "jsonld": "https://wpnews.pro/news/dualheads-reward-tilted-few-step-diffusion-made-simple.jsonld"}}