{"slug": "arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english", "title": "Arabic (Egyptian dialect) veterinary diagnosis assistant: dialect -> clinical-English gap and low accuracy on real owner messages", "summary": "A solo founder building an Arabic veterinary diagnosis assistant for livestock and pets in Egypt reported that retrieval-first matching of Egyptian colloquial owner messages against 530,000 chunks from roughly 216 English vet textbooks reached only about 40% exact agreement and about 72% exact-or-related agreement on real owner messages from Arabic Q&A sites. The founder measured that giving the LLM the correct passage yields about 92% top-1 accuracy, and that an LLM-proposes-10-candidate-diseases-then-search-by-disease-name pipeline reaches 92% top-3 on clean Modern Standard Arabic case descriptions, while LLM rewriting of questions into English clinical signs scored 38.3% versus 40.4% without and BGE-M3 was not better than paraphrase-multilingual-mpnet and ran 7x slower. The founder plans self-consistency over candidate lists, 2-3 follow-up questions to separate the top-3, a canonical disease list with hierarchical scoring, and a small paid vet audit.", "body_md": "I’m a solo founder building an Arabic veterinary diagnosis assistant for livestock and pets in Egypt (poultry, cattle, buffalo, horses, cats, dogs).\n\n**Setup:** ~216 English vet textbooks → 530k chunks (Postgres + pgvector, paraphrase-multilingual-mpnet 768-d with a 128-token window, plus lexical search); Claude Haiku as the LLM.\n\n**What we measured:**\n\n- Retrieval-first (embed the Arabic symptoms → top-10 chunks → LLM) is weak.\n- Given the correct passage, the LLM gets ~92% top-1, so the reasoning works.\n- “LLM proposes 10 candidate diseases → search the books by disease name → LLM picks top-3” reaches 92% top-3 on clean Arabic (MSA) case descriptions.\n- On real owner messages from Arabic Q&A sites, agreement is only ~40% exact and ~72% exact or related (labels are the replier’s answers, not vet-reviewed).\n\n**The translation/dialect problem:** owners write in Egyptian colloquial Arabic (“الفراخ بتنهج وفاتحة بوقها”, “الجاموسة مانعة الأكل”), while all the evidence is English and organised by disease name. What we tried:\n\n1. AGROVOC Arabic–English glossary: +1 case, and generic terms (“heat”, “symptoms”) diluted queries.\n2. A 287-term Egyptian field-term dictionary: fixed 3 cases, broke 3.\n3. A 410-term controlled clinical vocabulary for poultry (MSA + Egyptian): did not meet our gate.\n4. LLM rewriting of the whole question into English clinical signs before diagnosis: 38.3% vs 40.4% without, so no gain.\n5. BGE-M3 instead of mpnet: not better on our pilot, and 7x slower.\n\n**Planned next:** self-consistency over candidate lists, 2–3 follow-up questions to separate the top-3, a canonical disease list with hierarchical scoring, and a small paid vet audit.\n\n**Questions:**\n\n1. What has worked for you mapping dialectal or lay symptom text to clinical concepts? Is the right move to normalise the dialect, or to ask follow-up questions?\n2. Do you know Egyptian/Arabic medical NLP resources (dialect symptom lexicons, annotated corpora) or veterinary ontologies (VeNom, SNOMED vet) that can be used?\n3. Any pitfalls with LLM-simulated owners for evaluation?\n\nThanks!\n\n \n \nI think the biggest gap is embedding Egyptian Arabic symptoms straight against English textbook chunks. I’d try a step before retrieval: have the LLM rewrite the owner’s message into a short clinical English symptom list, then retrieve on that. 128 tokens is also pretty tight for textbook text, I’d test 300-500 side by side on your eval set. keeping each chunking + model variant as its own index on the same table makes those comparisons cheap: [github.com/pixeltable/pixeltable/blob/main/docs/release/platform/embedding-indexes.ipynb](https://github.com/pixeltable/pixeltable/blob/main/docs/release/platform/embedding-indexes.ipynb)", "url": "https://wpnews.pro/news/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english", "canonical_source": "https://discuss.huggingface.co/t/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english-gap-and-low-accuracy-on-real-owner-messages/190643#post_2", "published_at": "2026-10-16 00:43:21+00:00", "updated_at": "2026-10-16 00:59:41.007491+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "ai-research", "ai-products"], "entities": ["Egypt", "Claude Haiku", "Postgres", "pgvector", "paraphrase-multilingual-mpnet", "BGE-M3", "AGROVOC", "VeNom"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english", "markdown": "https://wpnews.pro/news/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english.md", "text": "https://wpnews.pro/news/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english.txt", "jsonld": "https://wpnews.pro/news/arabic-egyptian-dialect-veterinary-diagnosis-assistant-dialect-clinical-english.jsonld"}}