{"slug": "fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to", "title": "Fusion is not one-size-fits-all: Cross-Modal Representation Alignment for Time-to-Event Modeling", "summary": "Researchers introduced a foundation model-driven framework for cross-modal representation alignment between CT imaging and longitudinal EHR data, improving time-to-event prediction across tasks and institutions. Fusion strategies consistently boosted concordance index by 1.5-5.4% over unimodal baselines, with contrastive alignment providing the most robust improvements for pulmonary embolism mortality prediction. The study establishes task-aware multimodal alignment as a necessary design principle for robust generalization in clinical deployment.", "body_md": "arXiv:2606.15038v1 Announce Type: new\nAbstract: Accurate time-to-event (TTE) prediction from multimodal clinical data remains challenging due to modality imbalance and distribution shift. We introduce a foundation model-driven framework for cross-modal representation alignment between CT imaging and longitudinal EHR data, designed to generalize across tasks and institutions. CT and EHR modalities are encoded independently using domain-specific foundation models and aligned in a shared latent space through four principled fusion strategies: late fusion, contrastive alignment, cross-attention, and co-attention. We evaluate two clinically distinct TTE tasks: pulmonary embolism (PE) mortality and cardiovascular disease (CVD) outcomes, on large-scale multi-institutional cohorts (PE: N=3,099 train; 1,098 internal; 435 external; CVD: N=2,951 train; 837 internal; 682 external). Fusion consistently improves concordance index by 1.5-5.4% over unimodal baselines when modalities contribute comparably. Overall, contrastive multimodal fusion, particularly with CLMBR representations, provided the most consistent and statistically robust improvements, especially for PE mortality prediction. For MACE, cross-attention (one-hot) achieved the highest internal performance and image-guided co-attention achieved the best external performance. We therefore introduce a generalizable foundation model-based cross-modal alignment framework and provide the first systematic analysis of fusion behavior under modality imbalance in TTE prediction. Our results establish task-aware multimodal alignment as a necessary design principle for robust generalization and scalable clinical deployment.", "url": "https://wpnews.pro/news/fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to", "canonical_source": "https://arxiv.org/abs/2606.15038", "published_at": "2026-06-16 04:00:00+00:00", "updated_at": "2026-06-16 04:20:51.913461+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence", "ai-research", "computer-vision", "natural-language-processing"], "entities": ["CLMBR"], "alternates": {"html": "https://wpnews.pro/news/fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to", "markdown": "https://wpnews.pro/news/fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to.md", "text": "https://wpnews.pro/news/fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to.txt", "jsonld": "https://wpnews.pro/news/fusion-is-not-one-size-fits-all-cross-modal-representation-alignment-for-time-to.jsonld"}}