GenSyn10: A Multi-Generative AI Dataset For Benchmarking Image Classification

Researchers introduced GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images generated using three architecturally diverse models (FLUX.2-dev, HunyuanImage-3.0, and Qwen-Image-2512) to benchmark AI-generated image detection. Evaluation of 17 classification models showed that while fine-tuned detectors achieved 97-99.9% accuracy on seen generators, accuracy dropped to 79-96% on unseen generators, highlighting persistent out-of-distribution generalization limitations.

arXiv:2607.16283v1 Announce Type: new Abstract: The rapid advancement of generative AI has outpaced our ability to reliably detect its outputs, particularly when detectors encounter generators they have not seen before. We introduce GenSyn10, a CIFAR-10-aligned synthetic image dataset of 60,000 images 10 classes, 32$\times$32, 50k/10k split generated using three architecturally diverse state-of-the-art models: FLUX.2-dev Rectified Flow Transformer , HunyuanImage-3.0 MoE Transformer , and Qwen-Image-2512 Multimodal Diffusion Transformer , to advance research in AI-generated image detection. A central challenge in this domain is that detectors perform well on known generators but degrade on unseen ones. GenSyn10 addresses this limitation by curating data from multiple contemporary architectures under a standardized generation protocol, enabling controlled and systematic evaluation of out-of-distribution OOD generalization to novel generators. Images are generated using a template-based prompt engine and downsampled to ensure consistency. We evaluate 17 image classification models under a four-stage protocol: real-data baseline, zero-shot transfer, fine-tuning, and retention. Despite a measurable domain gap, CIFAR-10-trained models achieve up to 96.86\% zero-shot accuracy on GenSyn10, increasing to 99.88\% after fine-tuning. In binary real-vs-synthetic classification, fine-tuned models achieve 97-99.9\% accuracy on seen generators but drop to 79-96\% on images from an unseen generator, highlighting persistent limitations in OOD generalization. These results establish GenSyn10 as a controlled benchmark for studying synthetic image detection beyond single-generator settings, supporting research on robustness, domain adaptation, and cross-generator generalization.