Efficient and Training-Free Single-Image Diffusion Models

Researchers have developed a training-free diffusion model that generates new images matching the internal patch structure of a single reference image, eliminating the need for computationally expensive neural network training. The method uses a closed-form denoiser on the image's multi-scale patches to achieve state-of-the-art generation quality and diversity. The approach enables megapixel image generation in one second and gigapixel generation in minutes, with applications including text-guided stylization, image symmetrization, and retargeting.

arXiv:2606.04299v1 Announce Type: new Abstract: We consider the problem of generating images whose internal structure -- defined by the distribution of patches across multiple scales -- matches that of a single reference image. Recent approaches address this problem by training a diffusion model on a single image. But even in this setting, training is computationally expensive and requires hours of optimization. Instead, we model the image using a dataset of its patches at different scales. As this dataset is finite and the dimensionality of its patches is small, the score function for a noisy patch can be computed tractably using an optimal, closed-form denoiser, eliminating the need for neural network training. We integrate this patch-based denoiser into an efficient, training-free image diffusion model, and we describe how our method connects to classical patch-based image restoration techniques. Our approach achieves state-of-the-art generation quality and diversity compared to trained single-image diffusion models, and we demonstrate applications, including unconditional image generation, text-guided stylization, image symmetrization, and retargeting. Further, we show that our approach is compatible with latent space diffusion, and we show multiple additional acceleration techniques to achieve megapixel single-image generation in one second, and gigapixel generation in minutes.