Louis Blankemeier, Ashwin Kumar and Akshay S. Chaudhari's Stanford-led team published Merlin, a 3D vision-language foundation model for computed tomography, in a March 4 Nature paper that takes aim at one of radiology AI's practical gaps: most medical vision-language models have been built around 2D images and shorter text, while CT interpretation is volumetric, text-heavy and tied to patient history.
Merlin was trained on paired abdominal CT scans, diagnosis codes and radiology reports, using more than 6 million images from 15,331 CT scans, more than 1.8 million diagnosis codes and more than 6 million report tokens in the training set. The researchers evaluated the model on 6 task types and 752 individual tasks, including zero-shot findings classification, phenotype classification, image-report retrieval, 5-year chronic disease prediction, radiology report generation and 3D organ segmentation.