A photo of your face could tell doctors how long you have left to live.
The breakthrough is just the latest science fiction-adjacent development courtesy of AI’s immense capacity for crunching data. In the new study, published in Lancet Digital Health, researchers describe how they leveraged a novel AI model to predict mortality risk in cancer patients.
The tool, known as FAHR-FaceSurvival (Foundation Artificial Intelligence for Health Recognition), relies on an AI foundation model that the Mass General Brigham research team trained on over 40 million general photos of faces and a different set of over 30,000 facial photos of patients with cancer. The cancer patient’s facial images were attached to details about their medical outcomes, allowing researchers to test the model’s accuracy.
“FAHR-FaceSurvival advances the field of facial health recognition AI a step further by learning survival risk directly from facial photographs of patients with cancer,” study co-author Hugo Aerts said. “To our knowledge, this is the first demonstration that an AI model can directly predict cancer survival from a simple photograph.”
A high score from the FAHR-FaceSurvival tool was “significantly associated with early mortality risk,” according to the study’s results. Out of facial images that earned the highest 10% of scores, half of those 122 patients died within three months, compared with just 2% of those in the group with the lowest scores.
The cancer survival model focused specifically on the part of the face below the eyes and around the bridge of the nose, although researchers don’t fully understand what the AI system saw in those parts of the face.
Biological age is just a number
The research is the latest science to explore “facial biomarkers,” i.e. how a simple photo of the face can give doctors a surprisingly in-depth picture of a person’s health. But cancer survival rates won’t be the last thing scientists seek to predict using facial data.
“This foundation-model approach gives us a scalable way to develop multiple facial health biomarkers rather than training each model from scratch,” Aerts said.
The same team previously developed a parallel tool called FAHR-FaceAge that can predict a person’s biological age based on their facial characteristics. The research group found that the facial age tool offered a complementary picture of a person’s health when used alongside the cancer survival tool and improved the analysis for cancer survival rates.
Unlike the cancer survival prediction tool, that system fixated on the nasolabial folds (smile lines), the forehead above the eyes and the area around the temples. The system was consistent even when someone in a photo was wearing makeup, depicted in challenging lighting, and even if they had a nose job, according to the researchers.
Signs of facial aging reveal a lot about the way that someone lived, including their alcohol, drug use, and smoking habits. The research team found that smoking was the variable most positively correlated with facial aging, with current smokers appearing older than former smokers and people who had quit smoking looking older than non-smokers.
“As the population ages, physicians need better ways to estimate physical reserve and frailty before recommending intensive treatments such as radiation, surgery or systemic therapy,” one of the study’s authors Raymond Mak MD, an oncologist at Mass General Brigham Cancer Institute, said. “Our goal is to make face photograph-based health assessment more quantitative, consistent and scalable so it can support personalized treatment decisions rather than replace physician judgment.”