# Mammography Models: Are They Really Up to the Task? > Source: > Published: 2026-07-14 07:39:27+00:00 # Mammography Models: Are They Really Up to the Task? A look at how foundation models for mammography hold up under domain shifts. Are these models truly reliable, or is there more than meets the eye? Foundation models, touted as the future of AI, have been making waves in medical imaging, especially in mammography. But are they as strong as they claim when faced with new data? I ran the numbers so you don't have to. ## The Mammography Challenge Mammography isn't just any imaging task. It's life or death for many. Researchers recently put 15 foundation models under the microscope, testing them across breast density, BI-RADS severity, and cancer status. Sounds thorough, right? But here's the kicker: they trained these models on only three datasets and then unleashed them on 12 out-of-distribution (OOD) datasets. That's like [training](/glossary/training) for a marathon on a treadmill and then hitting rugged terrain. And the results? A mixed bag. Models like Mammo-FM and MaMA did show some promise, leading the charge in OOD performance. But don't pop the champagne just yet. Their success wasn't solely due to their mammography-specific training. Turns out, DINOv3, a vision-only model, held its ground surprisingly well. : is all that specialized pretraining really worth the hype? ## Generalization: The Achilles' Heel One would think that adapting models specifically for mammography would give an edge. But the results show otherwise. The attempt at mammography-adapted pretraining didn't exactly set the world on fire when it came to generalization. I tested this so you don't have to. Models still struggled to maintain consistent performance across various datasets. Digging deeper into the data reveals an unsettling truth. Even top-performing models displayed inconsistency across different datasets. It's like they're solid on some tracks but slip on others. That's a warning sign if there ever was one. If these models can't handle diverse data, how reliable are their clinical applications? ## What's in the Feature Space? Inspection of the feature space sheds light on this puzzle. While some models preserved key clinical signals, they also retained dataset biases. This dual nature hints at an underlying issue: models might not be as adaptable as we're led to believe. So where does this leave us? In a field where every pixel matters, is it wise to lean heavily on these foundation models? Perhaps it's time to rethink and overhaul how we evaluate these models, prioritizing OOD performance above all. If you're not running these tests locally yet, you're missing out on critical insights. Open weights don't wait for permission, and the speed difference isn't theoretical. You feel it. For the brave ones who want to dive into the code themselves, it's all out there. The research team was kind enough to share their codebase. It's on GitHub, just waiting for you to take it for a spin. Get AI news in your inbox Daily digest of what matters in AI.