Abstract
The choice between English-derived loanwords (gairaigo) and native Japanese equivalents is a socially meaningful aspect of language use, carrying implications for register, style, and pragmatic interpretation. We introduce a controlled evaluation dataset probing how large language models encode this form of sociolinguistic variation. The dataset comprises 113 interchangeable lexical pairs embedded across six communicative contexts spanning formal and informal, spoken and written registers. We evaluate 16 Japanese-capable LLMs across three complementary tasks: sentence rating, pairwise choice, and masked word prediction. Although both lexical forms were generally rated as natural, models diverged substantially in contextual sensitivity and lexical preference, revealing architectural differences in how socially grounded lexical alternatives are represented. These findings suggest that surface fluency may mask instability in modeling pragmatic variation, with implications for socially aware language generation and evaluation.- Anthology ID:
- 2026.nlpcss-1.2
- Volume:
[Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science](/volumes/2026.nlpcss-1/)- Month:
- July
- Year:
- 2026
- Address:
- San Diego
- Editors:
[Dallas Card](/people/dallas-card/),[Anjalie Field](/people/anjalie-field/),[Katherine Keith](/people/katherine-keith/),[Julia Mendelsohn](/people/julia-mendelsohn/)- Venues:
[NLP+CSS](/venues/nlpcss/)|[WS](/venues/ws/)- SIG:
- Publisher:
- Association for Computational Linguistics
- Note:
- Pages:
- 22–36
- Language:
- URL:
[https://aclanthology.org/2026.nlpcss-1.2/](https://aclanthology.org/2026.nlpcss-1.2/)- DOI:
[10.18653/v1/2026.nlpcss-1.2](https://doi.org/10.18653/v1/2026.nlpcss-1.2)- Cite (ACL):
- Joseph James. 2026. Borrowed Words, Borrowed Minds: Probing LLM Choice of English-Derived Loanwords in Japanese. InProceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science, pages 22–36, San Diego. Association for Computational Linguistics. - Cite (Informal):
[Borrowed Words, Borrowed Minds: Probing LLM Choice of English-Derived Loanwords in Japanese](https://aclanthology.org/2026.nlpcss-1.2/)(James, NLP+CSS 2026)- PDF:
[https://aclanthology.org/2026.nlpcss-1.2.pdf](https://aclanthology.org/2026.nlpcss-1.2.pdf)