Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis

Researchers at Zhejiang University introduced Holtercare-23K, a multimodal dynamic ECG dataset with 22,980 QA pairs from 788 clinical Holter records, and Holtercare-Bench, a benchmark evaluating MLLMs on temporal localization, clinical diagnosis, and global summarization. Zero-shot evaluations of leading MLLMs revealed significant performance gaps on ultra-long pathological sequences, but fine-tuning improved results. The project is available at https://github.com/ZJU4HealthCare/Holtercare-Bench.

arXiv:2608.19297v1 Announce Type: new Abstract: While multimodal large language models MLLMs excel in medical applications, most of them favor static images or short-term signals. In the critical field of dynamic electrocardiograms ECG , models struggle with complex temporal reasoning and diagnostic report generation due to a lack of high-quality datasets and benchmarks. To address this, we introduce i Holtercare-23K, a large-scale multimodal dynamic ECG dataset comprising 22,980 QA pairs derived from 788 clinical Holter records and featuring a novel signal-video-text tri-modal alignment. Based on this dataset, we present ii Holtercare-Bench, a multimodal benchmark that evaluates models on temporal localization, clinical diagnosis, and global summarization. Zero-shot evaluations of leading MLLMs reveal a significant performance gap in processing ultra-long pathological sequences. However, fine-tuning representative models yields substantial improvements. This work illuminates the limitations of current MLLMs in electrophysiology and provides a foundational benchmark for long-term medical MLLMs. Our project is available at https://github.com/ZJU4HealthCare/Holtercare-Bench.