cd /news/natural-language-processing/how-abjads-like-arabic-and-hebrew-le… · home topics natural-language-processing article
[ARTICLE · art-94350] src=dev.to ↗ pub= topic=natural-language-processing verified=true sentiment=· neutral

How Abjads Like Arabic and Hebrew Leave Vowels Ambiguous for AI

An engineer at Multigrid AI explains how abjad scripts like Arabic and Hebrew, which omit short vowels in everyday writing, create ambiguity for AI models trained on unvocalised text. The post details how consonantal roots such as k-t-b and s-p-r can map to multiple words, and notes that models must infer the intended reading from context, leading to errors in tasks requiring explicit vocalisation.

read4 min views1 publishedAug 12, 2026

An abjad writes consonants and leaves most vowels to the reader. That is not a deficiency of the script — it works because Semitic morphology puts the lexical meaning in the consonants — but it means an unvocalised written word frequently corresponds to several spoken words, and any system reading that word has to choose.

Arabic and Hebrew both work from a consonantal root, typically three consonants, carrying a broad semantic field. Vowel patterns applied to that root produce the specific words: agent, action, place, plural, passive, tense. In everyday writing the consonants are written and the vowel pattern is not.

Both scripts have full vowel notation available — Arabic harakat, Hebrew niqqud — and both restrict it to a specific set of contexts: sacred texts, poetry, children’s books, dictionaries, language teaching, and occasionally a single word in ordinary prose whose reading would otherwise be genuinely unclear. Newspapers, websites, contracts, chat messages and the overwhelming majority of the training corpus carry none. So a model trained on Arabic or Hebrew has learned from text where the vowels are absent, and asking it to produce vocalised output is asking it for something rare in its training distribution.

The Arabic root k-t-b

concerns writing. Written unvocalised as three letters, the string supports at least these readings:

كتب  (k-t-b) unvocalised
  kataba   he wrote            (past, active)
  kutiba   it was written      (past, passive)
  kutub    books               (plural noun)
  kutub    write! (dialectal / regional imperative forms vary)

Hebrew behaves the same way. The root s-p-r

concerns counting and recounting, and the unvocalised three-letter string supports:

ספר  (s-p-r) unvocalised
  sefer    a book
  sofer    a scribe / writer
  safar    he counted
  sapar    a barber
  sippur   a story            (with different vocalisation)

These are not obscure readings dug up to make a point — they are all common words. The reader disambiguates instantly from context and almost never notices doing it. The model does the same job, from the same information, with the same failure mode when context is thin.

One important qualification, because “abjads do not write vowels” is a useful simplification that is not quite true. Both scripts write long vowels using consonant letters pressed into service as vowel signs — the matres lectionis. Arabic uses alef, waw and ya for long a, u and i; Hebrew uses aleph, vav, yod and he the same way, and modern Israeli spelling (ktiv male) adds extra vav and yod precisely to reduce ambiguity in unvocalised text. So the ambiguity is partial rather than total: it is the short vowels, which carry most of the grammatical information, that go unwritten. That is why the surviving ambiguity falls so heavily on tense, voice and number rather than on which root is meant.

Not a rule, and not a lookup. The model is producing a distribution over continuations, so the reading it commits to is whichever is most probable given everything else in the context. In practice three things dominate:

A fourth factor is worth naming for Arabic specifically: the same consonantal string can be Modern Standard Arabic or one of several regional varieties with different vocalisations and different meanings. Dialect is another axis the model has to guess along, which is why regional dialect confusion and vowel ambiguity so often show up as one bug rather than two.

Generation is mostly fine, because the model produces unvocalised text and readers vocalise it as they always have. The failures cluster in tasks that need the vowels made explicit:

The practical pattern is the same one that works for tone marks: keep the original as written, and derive a normalised form for comparison. For Arabic, a standard normalisation removes harakat and the tatweel elongation character, and folds the alef variants and the final ya and alef maqsura to canonical forms. For Hebrew, strip niqqud and cantillation marks. Index the normalised form, store and display the original.

The Arabic normalisation is worth spelling out because it is where the duplicate-record bugs actually come from, and it is not the same operation as stripping vowels. Alef appears as bare alef, alef with hamza above, alef with hamza below and alef with madda, and writers use them inconsistently in informal text; final ya and alef maqsura are routinely interchanged in Egyptian usage; and ta marbuta and final ha are confused the same way. A normaliser that folds those families and removes tatweel will collapse most of the spelling variation for the same word without touching the letters that distinguish different words. Apply it symmetrically to the query and the index, and never to the stored original.

Where you need the vowels — speech, transliteration, teaching material — treat diacritisation as an explicit step whose output is a guess, not a fact. Give the model the largest context you have, since syntax and collocation are the only signals available to it, and never write the diacritised form back over the source. And in any interface where a single word arrives with no surrounding text, expect the training-frequency default and design for a correction rather than for being right.

── more in #natural-language-processing 4 stories · sorted by recency
── more on @multigrid ai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/how-abjads-like-arab…] indexed:0 read:4min 2026-08-12 ·