Spectral Guidance for Flexible and Efficient Control of Diffusion Models

Researchers introduced Spectral Guidance, a framework that controls diffusion models by exploiting the geometric structure of noise corruption to identify a small set of informative features. The method projects guidance signals directly onto the sampling trajectory, improving conditional accuracy on CIFAR-10 by 37 percentage points over training-free baselines while achieving four times faster sampling. The approach also enables spatial control without auxiliary models and identifies an optimal time window for effective guidance.

arXiv:2605.28900v1 Announce Type: new Abstract: We introduce Spectral Guidance, a framework for controlling diffusion models by leveraging the intrinsic geometry of the generative process. As data is progressively corrupted by noise, only a small number of features remain informative for control. We characterize them as the singular functions of a conditional expectation operator and show that they can be learned via a self-supervised objective. Once recovered, this basis enables the projection of arbitrary guidance signals, such as labels, CLIP embeddings, or masks, directly onto the sampling trajectory. This approach allows for stable, high-fidelity control without retraining or denoiser backpropagation during sampling. Empirically, we improve conditional accuracy on CIFAR-10 by 37 percentage points over the strongest training-free baseline while offering $4\times$ faster sampling. Moreover, the same representations that support label and CLIP guidance also enable spatial control, such as mask-based guidance, without auxiliary models. Finally, our framework reveals a phase transition in the generative process, pinpointing the optimal time window for effective guidance.