Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation

A study by Rebecca Knowles and Cyril Goutte, presented at the 17th Conference of the Association for Machine Translation in the Americas (AMTA) in Québec City, Canada, in August 2026, found that large language models (LLMs) produce more errors in French than in English for safety- and regulation-related queries in Canada. The researchers manually built English-French query pairs with gold standard answers and annotated LLM-generated answers, finding that a machine translation pipeline can mitigate some disparities but faces challenges from unreliable sources and technical terms.

Abstract We examine crosslingual performance disparities in large language models LLMs in the context of safety- and regulation-related queries in Canada. We manually build a set of English and French query pairs with gold standard answers and collect LLM-generated answers, which are manually annotated for correctness. We find that LLMs are more likely to produce errors in their answers in French than in English. We investigate a machine translation pipeline, translating the French query, producing an English LLM response, and translating the response back to French. We find that, while it can mitigate some of these performance disparities, additional challenges such as the reliability and language of the cited sources or technical terms greatly impact that mitigation strategy.- Anthology ID: - 2026.amta-research.9 - Volume: Proceedings of the 17th Conference of the Association for Machine Translation in the Americas Volume 1: Research Track /volumes/2026.amta-research/ - Month: - August - Year: - 2026 - Address: - Québec City, Canada - Editors: Eleftheria Briakou /people/eleftheria-briakou/unverified/ , Jeremy Gwinnup /people/jeremy-gwinnup/ , Shivali Goel /people/shivali-goel/unverified/ - Venue: AMTA /venues/amta/ - SIG: - Publisher: - Association for Machine Translation in the Americas - Note: - Pages: - 146–158 - Language: - URL: https://aclanthology.org/2026.amta-research.9/ https://aclanthology.org/2026.amta-research.9/ - DOI: - Cite ACL : - Rebecca Knowles and Cyril Goutte. 2026. Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation https://aclanthology.org/2026.amta-research.9/ . In Proceedings of the 17th Conference of the Association for Machine Translation in the Americas Volume 1: Research Track , pages 146–158, Québec City, Canada. Association for Machine Translation in the Americas. - Cite Informal : Crosslingual Disparities in LLM Performance: Challenges for MT as Mitigation https://aclanthology.org/2026.amta-research.9/ Knowles & Goutte, AMTA 2026 - PDF: https://aclanthology.org/2026.amta-research.9.pdf https://aclanthology.org/2026.amta-research.9.pdf