Text-to-Image Generative Modeling Without Language Pretraining

Researchers introduced Halcyon, a family of early-fusion multimodal autoregressive text-to-image models pretrained entirely from scratch without internet-scale language pretraining, challenging the assumption that language pretraining is necessary for strong text-conditioned image generation. The work directly asks whether language pretraining is required for strong T2I generation and reports results from models trained without it.

Modern text-to-image T2I systems use internet-scale language pretraining for text-conditioned image generation. We ask whether language pretraining is necessary for strong T2I generation. We introduce Halcyon, a family of early-fusion multimodal autoregressive models pretrained entirely from scrat