{"slug": "text-conditional-jepa-for-learning-semantically-rich-visual-representations", "title": "Text-Conditional JEPA for Learning Semantically Rich Visual Representations", "summary": "Researchers Chen Huang, Xianhang Li, Vimal Thilak, Etai Littwin, and Josh Susskind have developed Text-Conditional JEPA (TC-JEPA), a visual self-supervised learning model that uses image captions to reduce prediction uncertainty in masked feature learning. By modulating predicted patch features through sparse cross-attention over text tokens, the approach produces more semantically meaningful representations and improves downstream performance and training stability. TC-JEPA establishes a new vision-language pretraining paradigm based solely on feature prediction, outperforming contrastive methods on tasks requiring fine-grained visual understanding and reasoning.", "body_md": "[content type paper](/research/)published May 2026\n\nText-Conditional JEPA for Learning Semantically Rich Visual Representations\n\nAuthorsChen Huang, Xianhang Li, Vimal Thilak, Etai Littwin, Josh Susskind\n\nText-Conditional JEPA for Learning Semantically Rich Visual Representations\n\nAuthorsChen Huang, Xianhang Li, Vimal Thilak, Etai Littwin, Josh Susskind\n\nImage-based Joint-Embedding Predictive Architecture (I-JEPA) offers a promising approach to visual self-supervised learning through masked feature prediction. However with the inherent visual uncertainty at masked positions, feature prediction remains challenging and may fail to learn semantic representations. In this work, we propose Text-Conditional JEPA (TC-JEPA) that uses image captions to reduce the prediction uncertainty. Specifically, we modulate the predicted patch features using a fine-grained text conditioner that computes sparse cross-attention over input text tokens. With such conditioning, patch features become predictable as a function of text, thus are more semantically meaningful. We show TC-JEPA improves downstream performance and training stability, with promising scaling properties. TC-JEPA also offers a new vision-language pretraining paradigm based on feature prediction only, outperforming contrastive methods on diverse tasks, especially those requiring fine-grained visual understanding and reasoning.\n\nRethinking JEPA: Compute-Efficient Video SSL with Frozen Teachers\n\nOctober 8, 2025[research area Computer Vision](/research/?domain=Computer%20Vision), [research area Methods and Algorithms](/research/?domain=Methods%20and%20Algorithms)[conference ICLR](/research/?event=ICLR)\n\nVideo Joint Embedding Predictive Architectures (V-JEPA) learn generalizable off-the-shelf video representation by predicting masked regions in latent space with an exponential moving average (EMA)-updated teacher. While EMA prevents representation collapse, it complicates scalable model selection and couples teacher and student architectures. We revisit masked-latent prediction and show that a frozen teacher suffices. Concretely, we (i) train a…\n\nHow JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks\n\nNovember 20, 2024[research area Computer Vision](/research/?domain=Computer%20Vision), [research area Methods and Algorithms](/research/?domain=Methods%20and%20Algorithms)[conference NeurIPS](/research/?event=NeurIPS)\n\nTwo competing paradigms exist for self-supervised learning of data representations. Joint Embedding Predictive Architecture (JEPA) is a class of architectures in which semantically similar inputs are encoded into representations that are predictive of each other. A recent successful approach that falls under the JEPA framework is self-distillation, where an online encoder is trained to predict the output of the target encoder, sometimes using a…", "url": "https://wpnews.pro/news/text-conditional-jepa-for-learning-semantically-rich-visual-representations", "canonical_source": "https://machinelearning.apple.com/research/text-conditional-jepa-visual-representations", "published_at": "2026-05-07 00:00:00+00:00", "updated_at": "2026-05-29 08:05:08.438478+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "artificial-intelligence", "neural-networks", "ai-research"], "entities": ["Chen Huang", "Xianhang Li", "Vimal Thilak", "Etai Littwin", "Josh Susskind", "I-JEPA", "TC-JEPA"], "alternates": {"html": "https://wpnews.pro/news/text-conditional-jepa-for-learning-semantically-rich-visual-representations", "markdown": "https://wpnews.pro/news/text-conditional-jepa-for-learning-semantically-rich-visual-representations.md", "text": "https://wpnews.pro/news/text-conditional-jepa-for-learning-semantically-rich-visual-representations.txt", "jsonld": "https://wpnews.pro/news/text-conditional-jepa-for-learning-semantically-rich-visual-representations.jsonld"}}