Research into how AI can help users understand skin conditions

Google Research scientists Rory Sayres and Yun Liu published findings in JAMA Dermatology showing that AI tools can help laypeople better identify skin conditions and determine appropriate next steps. The study of 2,345 participants found that structured AI assistance significantly improved consumers' ability to name conditions compared to those using only internet searches or no tools. The research addresses the challenge that over half of adults seek health information online but often lack the medical knowledge to search effectively for specific skin concerns.

June 12, 2026 Rory Sayres and Yun Liu, Research Scientists, Google Research We present recent published findings on how dermatology AI tools may help laypeople with their own skin-related questions. More than half of adults https://www.cdc.gov/nchs/products/databriefs/db482.htm use the Internet for health information, and one-third https://www.kff.org/health-information-trust/poll-1-in-3-adults-are-turning-to-ai-chatbots-for-health-information-equaling-the-share-who-use-social-media-for-health/ turn to artificial intelligence AI . However, access to information does not mean that it is easy to understand https://research.google/blog/making-complex-text-understandable-minimally-lossy-text-simplification-with-gemini/ or correctly interpreted https://www.nature.com/articles/s41591-025-04074-y . In short, the human component of AI for health information remains important to research to help people benefit from better health information. Specifically, this is important in the space of dermatology skin, hair, nails; henceforth “skin” for brevity because people have trouble looking for the right information online related to their skin concern. For instance, you may notice “red dots on legs,” but not have the background knowledge to specifically search for “ palpable purpura https://en.wikipedia.org/wiki/Palpable purpura ”. Over the years, we have built a technical foundation in this area, including developing AI models https://research.google/blog/using-deep-learning-to-inform-differential-diagnoses-of-skin-diseases/ to inform differential diagnoses, performing validation of model generalization https://www.thelancet.com/journals/ebiom/article/PIIS2352-3964 25 00210-5/fulltext , and releasing datasets like SCIN https://blog.research.google/2024/03/scin-new-resource-for-representative.html to help clinicians and researchers. However, the most significant impact can only be realized by supporting the decision-making of people who have skin concerns through providing high-quality information. To do this right, understanding how humans engage with AI to inform their decisions is critical. Previous studies https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2777835 evaluating non-AI tools have shown that while people might get better at identifying a condition using the internet, they don't necessarily get better at deciding what next steps to take. We need to ensure that as AI tools become available, we carefully study and improve upon the human factors to support people in making better decisions. With the above in mind, today we share some of our recent and past research on consumer understanding of AI tools for their dermatology-related questions. These include a recent large-scale quantitative paper https://jamanetwork.com/journals/jama/fullarticle/10.1001/jamadermatol.2026.0597 that demonstrates increased ability to name conditions with AI assistance, as well as some benefits in determining what next steps to take. It also includes an in-depth mixed-methods study https://dl.acm.org/doi/10.1145/3706599.3720531 addressing how people use these tools on their own skin concerns, and how the understanding they gain compares to that from conversation with doctors. In “ Consumer Understanding of Skin Concerns With an AI-Powered Informational Tool https://jamanetwork.com/journals/jama/fullarticle/10.1001/jamadermatol.2026.0597 ,” published this week in JAMA Dermatology https://jamanetwork.com/journals/jamadermatology , we investigated how structured AI assistance changes a user's ability to identify a condition and determine their next steps. We showed 2,345 survey participants retrospective, de-identified skin condition cases — complete with images and structured medical history — and asked them to imagine the cases were their own. Participants were randomized into three groups to research the cases: We found that AI assistance provided a statistically significant improvement for consumer understanding. When using the AI tool, participants were more willing to attempt to name the condition shown over 62% compared to the control group using standard search tools 41% . More importantly, participants’ condition name guessing accuracy improved dramatically. Accuracy was nearly three times higher in the AI arm 23% compared to the unassisted control arm 8% . In the "Wizard of Oz" arm, accuracy was about four times higher 36% , but still not near perfect. Having AI "cards" to display matching conditions also imparted significantly higher confidence in their condition guesses, and greater overall satisfaction with their search results and the time spent searching. To avoid being prescriptive, the AI in our study was designed to focus on matching images to possible conditions and relying on the user to interpret what should be done. Our goal was to enable users to search efficiently and not to be prescriptive or diagnostic. In addition, the treatment and information given was written by dermatologists with access to authoritative sources, based purely on the condition name and not tailored to the specific severity of the condition in that case. Perhaps because of the generality of information provided, deciding on the appropriate medical next steps, such as using a home remedy versus scheduling an urgent clinic visit, remained challenging for users. Our study found that while next-step accuracy increased by a small amount in the "Wizard of Oz" arm 63.5% vs 60% in control , the standard AI arm did not show a statistically significant improvement. Furthermore, participants in the AI arm were slightly more likely to suggest a less urgent next step than a dermatologist would, compared to the control group 30% vs 27% . This reinforces that simply identifying the condition is not always enough. There is still progress to be made in designing tools that better inform laypeople about the safest and most appropriate next steps. While large-scale survey studies are invaluable for understanding general trends, we also recognized the need to understand how people interpret information when it is directly relevant to their own concerns, rather than interpreting pictures of others’ conditions. To get this richer, more nuanced feedback, we sought deep, qualitative insights directly from the communities who stand to benefit most from these tools. In " Navigating Skin Concerns with AI: A Human-Centered Investigation of a Dermatology App in a Diverse Community https://dl.acm.org/doi/10.1145/3706599.3720531 ," published in the ACM Computer-Human Interaction https://dl.acm.org/conference/chi CHI conference last year, we collaborated with the Stanford Healthcare AI Applied Research Team HEA 3RT https://med.stanford.edu/healthcare-ai.html and the Crucially, we wanted to ensure we were building for this community; since the participants spoke four primary languages, the AI application was translated into their respective languages. Volunteers or staff fluent in the respective language were also present to facilitate communication. In this real-world study, 110 consented participants used the app and consulted with a clinician immediately after to clarify any concerns . Similar to the survey study above, using the app increased these participants’ ability to name their condition an increase of 260%, though the correct guess rate was overall low . Participants heavily relied on visual matching of the textbook images to their condition, highlighting the importance of having images from a spectrum of skin tones, condition severities, and body parts to help them “pattern match”. The clinicians in the study felt the app’s predictions were generally 86% consistent with their own assessments of the condition. Because the participants could open the app during the clinician consultation, the clinicians were also able to use it as a shared reference point for discussion and facilitate patient-doctor conversation. The clinicians reported the app as a helpful tool 92% of the time. Our studies above focused on the use of image-based AI to help individuals with diverse backgrounds better understand skin conditions. Key findings for possible improvements include providing more "textbook" examples to guide user understanding and pattern matching, and including actionable information more specific to the actual user query as opposed to the conditions . Additionally, our research using image similarity based tools https://www.medrxiv.org/content/10.1101/2024.10.25.24316155v1 support that an image and text i.e., multimodal approach to AI-based skin condition information search is preferred by laypersons over using either alone. When we look at all of these studies collectively, a potential picture of the future of searching for skin condition information emerges. Providing a visual start lowers the barrier to entry, and more personalized AI guidance may help navigate complex medical information. However, building highly effective tools requires continuous, human-centered research to ensure that everyone can effectively interpret this information to help support healthcare journeys. We thank Elyse Bagley, Trevor Crowell, Bhavna Daryani, Huy Doan, Morgan Du, Madison Elliott, Bea Erickson, Mat Fleck, Zoe Gan, Tammi Huynh, Yetunde Ibitoye, Yejin Jeong, Sergio Marquez, Jay Nayar, Kira Nguyen, Trang Nguyen, Javier Perez, Carola Ponce, Uriel Rivera, Sunny Virmani, Renee Wong, and Allan Ysunza for their contributions to the execution of the mixed-methods studies; and Michael Howell, Naama Hammel, Rajeev Rikhye, Abi Jones and Dave Steiner for valuable feedback on the papers.