Zoom In Disparities in Healthcare LLM Q&A

Researchers found substantial cross-lingual disparities in healthcare LLM Q&A, with models aligning more with English Wikipedia even for non-English prompts. Providing non-English contextual excerpts improved factual alignment toward culturally relevant knowledge, highlighting pathways for more equitable multilingual AI systems.

arXiv:2510.17476v2 Announce Type: replace Abstract: Equitable access to reliable health information is vital when integrating AI into healthcare. Yet, information quality varies across languages, raising concerns about the reliability and consistency of multilingual Large Language Models LLMs . We systematically examine cross-lingual disparities in pre-training source and factuality alignment in LLM answers for multilingual healthcare Q&A across English, German, Turkish, Chinese Mandarin , and Italian. We i constructed Multilingual Wiki Health Care MultiWikiHealthCare , a multilingual dataset from Wikipedia; ii analyzed cross-lingual healthcare coverage; iii assessed LLM response alignment with these references; and iv conducted a case study on factual alignment through the use of contextual information and Retrieval-Augmented Generation RAG . Our findings reveal substantial cross-lingual disparities in both Wikipedia coverage and LLM factual alignment. Across LLMs, responses align more with English Wikipedia, even when the prompts are non-English. Providing contextual excerpts from non-English Wikipedia at inference time effectively shifts factual alignment toward culturally relevant knowledge. These results highlight practical pathways for building more equitable, multilingual AI systems for healthcare.