arXiv:2611.23155v1 Announce Type: cross Abstract: Recent advances in text-to-image generation have substantially improved image quality, particularly through diffusion-based methods that require multiple inference steps. In contrast, Sphere Encoder offers an efficient generative framework capable of producing images with fewer forward steps. However, extending it to free-form text is nontrivial, because its class-conditioned design lacks cross-modal alignment. To address this challenge, we propose TextSphere, an efficient text-to-image generative framework that extends Sphere Encoder with language conditioning and cross-modal alignment. We incorporate pretrained CLIP text embeddings through adaptive layer normalization (AdaLN) to guide image generation while retaining the original spherical sampling mechanism. To bridge the cross-modal representation gap, we introduce a Spherical Semantic Alignment and Uniformity (SSAU) objective that jointly promotes semantic alignment and representation uniformity on the hypersphere. Experiments on CUB-200-2011, MS-COCO-2014, and Flickr30k demonstrate that TextSphere consistently reduces generation FID (gFID) over the text-conditioned baseline while achieving reconstruction FID (rFID) below 1.0. With only 4 and 6 sampling steps, TextSphere achieves lower gFID than the compared diffusion- and GAN-based models trained from scratch on CUB-200-2011 and MS-COCO-2014, respectively. We further observe that improved reconstruction fidelity does not necessarily yield better generation quality. In addition, we use Partial Least Squares Singular Value Decomposition (PLS-SVD) to analyze cross-modal alignment and modality gap, offering insight into the effectiveness of our approach.
TextSphere: Efficient Text-to-Image Generation via Shared Hyperspherical Alignment and Uniformity
TextSphere, a text-to-image framework extending Sphere Encoder with language conditioning and cross-modal alignment, reduces generation FID (gFID) over the text-conditioned baseline while achieving reconstruction FID (rFID) below 1.0, according to the arXiv paper 2611.23155v1. The framework feeds pretrained CLIP text embeddings through adaptive layer normalization (AdaLN) to guide image generation while retaining Sphere Encoder's spherical sampling mechanism, and adds a Spherical Semantic Alignment and Uniformity (SSAU) objective to bridge the cross-modal representation gap. On CUB-200-2011 and MS-COCO-2014, TextSphere reached lower gFID than diffusion- and GAN-based models trained from scratch using only 4 and 6 sampling steps respectively, and the authors used Partial Least Squares Singular Value Decomposition (PLS-SVD) to analyze cross-modal alignment and the modality gap.
Track generation FID and reconstruction FID separately when evaluating text-to-image models; the authors found rFID below 1.0 did not guarantee better gFID.
more in playbook: Evals and testing for LLM appsRun your AI side-project on zahid.host
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.