{"slug": "bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at", "title": "BPE vs. Morphology-Aware Tokenization: A Case Study on Arabic Language Modeling at BabyLM Scale", "summary": "A BabyLM 2026 workshop paper by Mohamed Badawy, Marion Di Marco and Alexander Fraser found no consistent downstream accuracy gains from morphology-aware tokenization over standard Byte Pair Encoding for Arabic language modeling at small scale. The authors propose Morfessor+MorphBPE, an unsupervised tokenizer that avoids the supervised Farasa segmenter barred by BabyLM 2026 rules against off-the-shelf linguistic tools; unsupervised Morfessor boundaries matched supervised Farasa on aggregate and scored higher on grammaticality (MultiBLiMP). Holding Farasa segmentation fixed, the MorphBPE constraint sharply lowered perplexity but barely changed downstream accuracy, and all tokenizer variants clustered within a narrow accuracy band, with gains appearing on language understanding rather than knowledge tasks.", "body_md": "##### Abstract\n\nByte Pair Encoding (BPE) is blind to morphology, a poor fit for Arabic, whose words bundle conjunctions, articles, and inflection into one token. The leading morphology-aware strategy for Arabic, MorphBPE, relies on Farasa, a supervised segmenter disallowed under the BabyLM 2026 rule against off-the-shelf linguistic tools. We study Arabic segmentation under the BabyLM GPT-2 training pipeline and propose Morfessor+MorphBPE, an unsupervised morphology-aware tokenizer.\nWe compare two dimensions: supervised vs. unsupervised morphological segmentation and the merging constraints used by BPE. All tokenizer variants cluster within a narrow accuracy band, with improvements rather on language understanding tasks than knowledge tasks, even though we do not observe consistent improvements over BPE. Furthermore, we find that holding the Farasa segmentation fixed, the MorphBPE constraint sharply lowers perplexity but barely changes downstream accuracy; and unsupervised Morfessor boundaries match supervised Farasa on aggregate while scoring higher on grammaticality (MultiBLiMP). Morfessor+MorphBPE thus offers a practical path to morphology-aware tokenization for Arabic.\n\n- Anthology ID:\n- 2026.babylm-main.16\n- Volume:\n- [Proceedings of the Second BabyLM Workshop](https://aclanthology.org/volumes/2026.babylm-main/)\n- Month:\n- October\n- Year:\n- 2026\n- Address:\n- Budapest, Hungary\n- Editors:\n- [Leshem Choshen](https://aclanthology.org/people/leshem-choshen/) ,[Ryan Cotterell](https://aclanthology.org/people/ryan-cotterell/) ,[Mustafa Omer Gul](https://aclanthology.org/people/mustafa-omer-gul/) ,[Jaap Jumelet](https://aclanthology.org/people/jaap-jumelet/unverified/) ,[Tal Linzen](https://aclanthology.org/people/tal-linzen/) ,[Aaron Mueller](https://aclanthology.org/people/aaron-mueller/) ,[Suchir Salhan](https://aclanthology.org/people/suchir-salhan/) ,[Raj Sanjay Shah](https://aclanthology.org/people/raj-sanjay-shah/) ,[Alex Warstadt](https://aclanthology.org/people/alex-warstadt/) ,[Ethan Gotlieb Wilcox](https://aclanthology.org/people/ethan-wilcox/)\n- Venues:\n- [BabyLM](https://aclanthology.org/venues/babylm/) |[WS](https://aclanthology.org/venues/ws/)\n- SIG:\n- Publisher:\n- Association for Computational Linguistics\n- Note:\n- Pages:\n- 199–208\n- Language:\n- URL:\n- [https://aclanthology.org/2026.babylm-main.16/](https://aclanthology.org/2026.babylm-main.16/)\n- DOI:\n- Cite (ACL):\n- Mohamed Badawy, Marion Di Marco, and Alexander Fraser. 2026. [BPE vs. Morphology-Aware Tokenization: A Case Study on Arabic Language Modeling at BabyLM Scale](https://aclanthology.org/2026.babylm-main.16/) . In*Proceedings of the Second BabyLM Workshop* , pages 199–208, Budapest, Hungary. Association for Computational Linguistics.\n- Cite (Informal):\n- [BPE vs. Morphology-Aware Tokenization: A Case Study on Arabic Language Modeling at BabyLM Scale](https://aclanthology.org/2026.babylm-main.16/) (Badawy et al., BabyLM 2026)\n- PDF:\n- [https://aclanthology.org/2026.babylm-main.16.pdf](https://aclanthology.org/2026.babylm-main.16.pdf)", "url": "https://wpnews.pro/news/bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at", "canonical_source": "https://aclanthology.org/2026.babylm-main.16/", "published_at": "2026-10-24 00:00:00+00:00", "updated_at": "2026-10-25 21:41:18.761573+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "ai-research", "machine-learning"], "entities": ["Mohamed Badawy", "Marion Di Marco", "Alexander Fraser", "Morfessor", "MorphBPE", "Farasa", "BabyLM", "Association for Computational Linguistics"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at", "markdown": "https://wpnews.pro/news/bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at.md", "text": "https://wpnews.pro/news/bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at.txt", "jsonld": "https://wpnews.pro/news/bpe-vs-morphology-aware-tokenization-a-case-study-on-arabic-language-modeling-at.jsonld"}}