Controllable Diverse Sampling through Pullback Directions

A training-free method that steers diffusion sampling trajectories along pullback-geometry directions of maximal output sensitivity improves sample diversity in prompt-conditioned image generation and inpainting, according to arXiv paper 2610.14294v1. The approach, tested on MS-COCO and BrushNet, increases variation across random seeds without additional optimization or sampling overhead while maintaining prompt alignment. The authors say existing diversity methods either add sampling cost or introduce distortions that degrade image quality.

arXiv:2610.14294v1 Announce Type: new Abstract: Conditional diffusion models can generate high-quality images that align closely with a given prompt. However, samples generated from different random seeds may remain visually similar, limiting creative exploration. Existing approaches can increase variation, but often require additional optimization or sampling overhead, while stronger perturbations may introduce distortions that degrade image quality or prompt alignment. We propose a training-free method that leverages the local pullback geometry induced by the mapping from the conditioning-embedding space to the image space to identify directions of maximal output sensitivity. By steering different sampling trajectories along these directions, we encourage distinct generations while maintaining consistency with the conditioning signal. Experiments on MS-COCO and BrushNet show the efficacy of the method in improving diversity on prompt-conditioned image generation and inpainting.