{"slug": "no-channels-left-behind-understanding-diffusion-acceleration-through-residual", "title": "No Channels Left Behind: Understanding Diffusion Acceleration through Residual Gates", "summary": "A new arXiv paper (2610.40914v1) argues that diffusion-transformer training acceleration from methods such as REPA and dispersive regularization stems at least partly from reorganizing the AdaLN-Zero residual gate α rather than solely from the auxiliary supervision or feature geometry those methods impose. The authors introduce the normalized participation ratio (nPR) to measure channel-wise gate organization and report that broader gate participation tracks better generation quality across REPA teachers, projector architectures and radial-objective targets; on ImageNet 256×256 with SiT-B/2, their teacher-free radial objective improves FID over dispersive loss while increasing mean gate participation over timesteps and classes.", "body_md": "arXiv:2610.40914v1 Announce Type: new \nAbstract: Training acceleration methods for diffusion transformers, such as REPA and dispersive regularization, are commonly explained by additional supervision, spatial-structure priors, or improved feature geometry. We revisit this view and uncover a striking internal change: the AdaLN-Zero residual gate $\\alpha$, which directly controls how conditioned updates enter the residual stream, is substantially reorganized by these methods. This observation motivates our hypothesis that acceleration arises, at least in part, from reshaping the residual gating pathway rather than solely from the feature structures explicitly imposed by auxiliary objectives. Supporting this view, accelerated models are markedly more sensitive to perturbations of $\\alpha$, and their gate geometry reflects semantic relationships among classes. We further introduce the normalized participation ratio (nPR) to characterize the channel-wise organization of the gates and find that, across REPA teachers, projector architectures, and radial-objective targets, broader gate participation accompanies better generation quality. Guided by this insight, we propose a simple teacher-free radial objective that promotes broader gate participation through feature-norm regularization, without pairwise feature comparisons. On ImageNet $256\\times256$ with SiT-B/2, our method improves FID over dispersive loss while increasing mean gate participation over timesteps and classes, further supporting the link between gate organization and training acceleration. Our results identify residual-gate reorganization as a measurable axis of diffusion training acceleration and suggest a simple direction for further acceleration.", "url": "https://wpnews.pro/news/no-channels-left-behind-understanding-diffusion-acceleration-through-residual", "canonical_source": "https://arxiv.org/abs/2610.40914", "published_at": "2026-11-02 05:00:00+00:00", "updated_at": "2026-11-02 05:21:13.219406+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "generative-ai", "ai-research", "neural-networks"], "entities": ["REPA", "SiT-B/2", "ImageNet", "AdaLN-Zero", "normalized participation ratio"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/no-channels-left-behind-understanding-diffusion-acceleration-through-residual", "markdown": "https://wpnews.pro/news/no-channels-left-behind-understanding-diffusion-acceleration-through-residual.md", "text": "https://wpnews.pro/news/no-channels-left-behind-understanding-diffusion-acceleration-through-residual.txt", "jsonld": "https://wpnews.pro/news/no-channels-left-behind-understanding-diffusion-acceleration-through-residual.jsonld"}}