Personalized Generative Models for Contextual Debiasing

Researchers have developed DecoupleGen, a method that uses personalized text-to-image diffusion models to generate training images of objects in uncommon contexts, such as a beach ball on a road. The approach aims to correct biases in vision datasets where common visual patterns are overrepresented, improving model recognition of objects in rare but important scenarios. Experiments on complex scene datasets show consistent performance gains over existing debiasing techniques.

arXiv:2605.26353v1 Announce Type: new Abstract: Different visual patterns appear with different frequencies in the world: e.g., beach balls appear on sand more often than they do on a road. These statistics are reflected in vision datasets, and as a result trained models more easily recognize objects in common scenarios. However, recognizing a beach ball on a road may arguably be even more important than recognizing it on sand. We study how to mitigate this discrepancy. Since collecting uncommon images in the real world may be difficult, we explore whether generating images with less frequent contexts can serve as effective training augmentation. A key challenge is guiding generations to remain close to the original dataset distribution while creating diverse images with uncommon contexts. We introduce Decoupling Contextual Patterns with Generations DecoupleGen , a method that personalizes text-to-image diffusion models to facilitate coherent synthesis of images with rare contexts while preserving original visual details. The generated images contain semantically meaningful content and remain visually aligned with the original datasets. We further apply verification constraints to ensure relevance of the augmented data. We evaluate our approach on object classification and recognition tasks on complex scene datasets. Our experiments demonstrate consistent improvements over previous approaches, and our analyses identify factors underlying these improvements.