Borrowed Words, Borrowed Minds: Probing LLM Choice of English-Derived Loanwords in Japanese

Researchers introduced a controlled evaluation dataset probing how large language models encode sociolinguistic variation in Japanese, specifically the choice between English-derived loanwords and native equivalents. Testing 16 Japanese-capable LLMs across three tasks revealed substantial divergence in contextual sensitivity and lexical preference, indicating that surface fluency may mask instability in modeling pragmatic variation.

Abstract The choice between English-derived loanwords gairaigo and native Japanese equivalents is a socially meaningful aspect of language use, carrying implications for register, style, and pragmatic interpretation. We introduce a controlled evaluation dataset probing how large language models encode this form of sociolinguistic variation. The dataset comprises 113 interchangeable lexical pairs embedded across six communicative contexts spanning formal and informal, spoken and written registers. We evaluate 16 Japanese-capable LLMs across three complementary tasks: sentence rating, pairwise choice, and masked word prediction. Although both lexical forms were generally rated as natural, models diverged substantially in contextual sensitivity and lexical preference, revealing architectural differences in how socially grounded lexical alternatives are represented. These findings suggest that surface fluency may mask instability in modeling pragmatic variation, with implications for socially aware language generation and evaluation.- Anthology ID: - 2026.nlpcss-1.2 - Volume: Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science /volumes/2026.nlpcss-1/ - Month: - July - Year: - 2026 - Address: - San Diego - Editors: Dallas Card /people/dallas-card/ , Anjalie Field /people/anjalie-field/ , Katherine Keith /people/katherine-keith/ , Julia Mendelsohn /people/julia-mendelsohn/ - Venues: NLP+CSS /venues/nlpcss/ | WS /venues/ws/ - SIG: - Publisher: - Association for Computational Linguistics - Note: - Pages: - 22–36 - Language: - URL: https://aclanthology.org/2026.nlpcss-1.2/ https://aclanthology.org/2026.nlpcss-1.2/ - DOI: 10.18653/v1/2026.nlpcss-1.2 https://doi.org/10.18653/v1/2026.nlpcss-1.2 - Cite ACL : - Joseph James. 2026. Borrowed Words, Borrowed Minds: Probing LLM Choice of English-Derived Loanwords in Japanese https://aclanthology.org/2026.nlpcss-1.2/ . In Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science , pages 22–36, San Diego. Association for Computational Linguistics. - Cite Informal : Borrowed Words, Borrowed Minds: Probing LLM Choice of English-Derived Loanwords in Japanese https://aclanthology.org/2026.nlpcss-1.2/ James, NLP+CSS 2026 - PDF: https://aclanthology.org/2026.nlpcss-1.2.pdf https://aclanthology.org/2026.nlpcss-1.2.pdf