Information-theoretic Multimodal Representation Learning for Electrocardiogram Signals

Researchers have developed MERIT, a multimodal pretraining framework that uses information theory to learn electrocardiogram (ECG) representations by jointly preserving signal structure and integrating clinical semantics. The method outperformed prior approaches, achieving over 3% F1 improvement on PTB-XL All and 5% F1 on SubClass classification, while also improving zero-shot performance by up to 2.66% AUC and 2.11% F1. The framework's learned representations further enhanced ECG-conditioned clinical text generation with large language models, demonstrating more informative and clinically meaningful outputs for fine-grained applications.

arXiv:2605.27583v1 Announce Type: new Abstract: Electrocardiograms ECGs are widely used non-invasive measurements of cardiac activity and play a central role in clinical diagnosis. Recent multimodal approaches align ECG signals with clinical reports to incorporate diagnostic semantics, but clinical reports often fail to preserve the rich physiological structure of ECG waveforms, particularly across multiple levels of abstraction ranging from coarse diagnostic categories to fine-grained morphology. To address this limitation, we formulate ECG representation learning from an information-theoretic perspective and derive a tractable objective that jointly preserves signal structure and integrates clinical semantics. Based on this principle, we propose \textbf{MERIT} Multimodal ECG Representation via Information Theory , a dual-branch pretraining framework combining masked ECG modeling with ECG--text contrastive alignment. Extensive experiments on PTB-XL and additional benchmarks demonstrate consistent improvements over prior methods, including gains exceeding $3%$ F1 on PTB-XL All and $5%$ F1 on SubClass classification. In zero-shot evaluation, MERIT further improves performance by up to $ +2.66\%$ AUC and $ +2.11\%$ F1 on PTB-XL SubClass, while also demonstrating robustness under multiple distribution-shift settings. Moreover, leveraging the learned ECG representations for ECG-conditioned clinical text generation with large language models improves text quality across several metrics, including ROUGE and METEOR. Together, these results demonstrate that MERIT learns more informative and clinically meaningful ECG representations, particularly for fine-grained clinical applications.