# Byte-pair Encoding Captures Aspects of Word Meaning that Distributional Semantics Overlooks

> Source: <https://aclanthology.org/2026.tacl-1.113/>
> Published: 2026-10-07 00:00:00+00:00

##### Abstract

Large language models learn word meanings through distributional semantics (i.e., patterns in text), and distribution does not always capture information about taxonomy and function, which are essential to human representations of word meaning. However, sound and spelling often conveys such information, and language models decompose many words into subword tokens, which may identify category markers. For example, GPT-5 segments fluorine and bromine into fluor + ine and brom + ine, which share the category marker ine. We provide evidence that, across 18 languages, shared tokens predict greater semantic priming effects in humans (e.g., people recognize bromine faster after reading fluorine), beyond what distributional similarity explains. Furthermore, across seven languages, shared tokens boost semantic priming beyond what shared morphemes explain. This suggests that subword tokens could help language models arrive at human-like representations of word meanings. However, we also provide evidence that the trend towards larger vocabularies of tokens obscures category markers, and even when language models have access to subword tokens, they underestimate the semantic relatedness of words that share tokens.
- Anthology ID:
- 2026.tacl-1.113
- Volume:
- [Transactions of the Association for Computational Linguistics, Volume 14](https://aclanthology.org/volumes/2026.tacl-1/)
- Month:
- Year:
- 2026
- Address:
- Cambridge, MA
- Venue:
- [TACL](https://aclanthology.org/venues/tacl/)
- SIG:
- Publisher:
- MIT Press
- Note:
- Pages:
- 2425–2448
- Language:
- URL:
- [https://aclanthology.org/2026.tacl-1.113/](https://aclanthology.org/2026.tacl-1.113/)
- DOI:
- [10.1162/tacl.a.810](https://doi.org/10.1162/tacl.a.810)
- Cite (ACL):
- David A. Haslett, Antoni B. Chan, and Janet H. Hsiao. 2026. [Byte-pair Encoding Captures Aspects of Word Meaning that Distributional Semantics Overlooks](https://aclanthology.org/2026.tacl-1.113/) .*Transactions of the Association for Computational Linguistics* , 14:2425–2448.
- Cite (Informal):
- [Byte-pair Encoding Captures Aspects of Word Meaning that Distributional Semantics Overlooks](https://aclanthology.org/2026.tacl-1.113/) (Haslett et al., TACL 2026)
- PDF:
- [https://aclanthology.org/2026.tacl-1.113.pdf](https://aclanthology.org/2026.tacl-1.113.pdf)