{"slug": "ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual", "title": "AC-JEPA: Anchored Cross-Modal Joint Embedding Predictive Architecture for Visual-to-Symbolic Translation", "summary": "A new arXiv paper (2610.36493v1) introduces Anchored Cross-Modal JEPA (AC-JEPA), an SFT-compatible plug-in that predicts masked intermediate states in both cross-modal directions to improve visual-to-symbolic translation. On optical chemical structure recognition, AC-JEPA trained on clean pairs reached 76.07% accuracy on the Simple set of the WildMol benchmark versus 67.40% for supervised fine-tuning alone, and AC-JEPA-based self-training on pseudo-labeled data raised accuracy to 80.23% while direct SFT fell from 67.40% to 63.47%. The authors present SFT-anchored joint-embedding prediction as an effective strategy for cross-modal translation under both curated and imperfect supervision.", "body_md": "arXiv:2610.36493v1 Announce Type: cross \nAbstract: Cross-modal translation requires models to capture shared semantics across modalities while preserving the fine-grained structure required by the target modality. Yet in real-world scientific domains, paired data are often imperfect, while obtaining clean pairs at scale is costly. Supervised fine-tuning (SFT) directly optimizes cross-modal translation through categorical token-level supervision, but this objective is sensitive to errors in the target sequences. JEPA offers a complementary representation-space objective, but most existing formulations focus on prediction within a modality rather than translation across modalities. We introduce Anchored Cross-Modal JEPA (AC-JEPA), an SFT-compatible plug-in that predicts masked intermediate states in both cross-modal directions using paired data. The joint-embedding objective targets cross-modal structure predictable from context, while the causal SFT path retains direct token-level supervision for the target sequence. During self-training, AC-JEPA learns from continuous latent targets derived from imperfect pseudo-labeled pairs, offering intrinsic tolerance to local data errors without promoting pseudo-tokens to hard categorical supervision. Using optical chemical structure recognition as a testbed, AC-JEPA trained on clean pairs achieves 76.07% accuracy on the Simple set of the WildMol benchmark, compared with 67.40% for SFT alone. Using the same pseudo-labeled data for both approaches, AC-JEPA-based self-training further raises accuracy to 80.23%, whereas accuracy under direct SFT drops from 67.40% to 63.47%. These results support SFT-anchored joint-embedding prediction as an effective strategy for cross-modal translation under both curated and imperfect supervision.", "url": "https://wpnews.pro/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual", "canonical_source": "https://www.machinebrief.com/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-arch-758x", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 05:41:53.005339+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "computer-vision", "artificial-intelligence"], "entities": ["AC-JEPA", "WildMol", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual", "markdown": "https://wpnews.pro/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual.md", "text": "https://wpnews.pro/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual.txt", "jsonld": "https://wpnews.pro/news/ac-jepa-anchored-cross-modal-joint-embedding-predictive-architecture-for-visual.jsonld"}}