Stanford's Merlin puts vision-language AI on full 3D CT scans

Stanford researchers led by Louis Blankemeier, Ashwin Kumar and Akshay S. Chaudhari published Merlin, a 3D vision-language foundation model for CT scans, in Nature on March 4. The model was trained on over 6 million images from 15,331 CT scans and evaluated on 752 tasks including disease prediction and report generation, addressing the gap of 2D-focused medical AI models.

Louis Blankemeier, Ashwin Kumar and Akshay S. Chaudhari's Stanford-led team published Merlin, a 3D vision-language foundation model for computed tomography, in a March 4 Nature paper https://www.nature.com/articles/s41586-026-10181-8 that takes aim at one of radiology AI's practical gaps: most medical vision-language models have been built around 2D images and shorter text, while CT interpretation is volumetric, text-heavy and tied to patient history. Merlin was trained on paired abdominal CT scans, diagnosis codes and radiology reports, using more than 6 million images from 15,331 CT scans, more than 1.8 million diagnosis codes and more than 6 million report tokens in the training set. The researchers evaluated the model on 6 task types and 752 individual tasks, including zero-shot findings classification, phenotype classification, image-report retrieval, 5-year chronic disease prediction, radiology report generation and 3D organ segmentation.