Representation-Conditioned Diffusion Models for Guided Training Data Generation

Researchers have developed representation-conditioned diffusion models that generate synthetic training images by conditioning on learned representations from DINOv2, DINOv3, and CLIP, achieving a 10.76 percentage point improvement in top-1 accuracy over class-conditioned generation on ImageNet100. Scaling the synthetic dataset allowed a classifier to outperform one trained on real data by 2.0 percentage points. The findings suggest that this approach could augment, complement, or replace real-world datasets in large-scale visual learning tasks.

arXiv:2605.27495v1 Announce Type: new Abstract: Data availability remains a critical bottleneck in many deep learning applications. Large-scale datasets are often expensive to collect, curate and annotate, which can limit the scalability and applicability of supervised learning methods. In this work, we evaluate the classification performance of models trained on synthetic image datasets produced by generative deep learning. In particular, we use latent diffusion models conditioned on learned representations from DINOv2, DINOv3, and CLIP. Our results demonstrates that this representation-conditioned formulation significantly outperforms class-conditioned generation by a large margin +10.76 p.p. top-1 accuracy on ImageNet100 , by improving sample quality and mode coverage. Furthermore, by scaling the size of the synthetic dataset, we are able to outperform a classifier trained on the real data +2.0 p.p top-1 accuracy . We also demonstrate how generated images can be used for augmentation purposes, outperforming classical augmentation methods, and how the conditioning space can be used for sample filtering to further improve training value. Collectively, these findings highlight that representation-conditioned diffusion models provide a promising approach for augmenting, complementing, or potentially replacing real-world datasets in large-scale visual learning tasks.