{"slug": "ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken", "title": "Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation", "summary": "Ahmed Ayman Ahmed Ezzat Mohamed's system ranked first on the English track and second on Modern Standard Arabic in the ImageEval 2026 Subtask 1a spoken visual QA task, scoring 0.962 on English and 0.834 on MSA against organizer baselines of 0.664 and 0.398, according to the paper published in the Proceedings of the Fourth Arabic Natural Language Processing Conference (ArabicNLP 2026, pages 711–720, Budapest). On Task 2 (CRAI-Bench), Mohamed's judge ensemble reached a test Spearman of 0.673 versus 0.533 for a reproduction of the organizer baseline. The paper attributes the margins to answer-token bias, Arabic speech recognition isolated by the parallel track design, and tie-breaking via a second judge, and reports a post-submission study finding a 9% answer-flip rate between identical runs at temperature 0.", "body_md": "##### Abstract\n\nWe describe our systems for both ImageEval 2026 shared tasks. On Subtask 1a (spoken visual QA) we reach 0.962 on English and 0.834 on Modern Standard Arabic against baselines of 0.664 and 0.398, ranking first on the English track and second on MSA, while on Task 2 (CRAI-Bench) our judge ensemble reaches a test Spearman of 0.673 against 0.533 for our reproduction of the organizer baseline. Three effects account for most of these margins: answer-token bias (digit labels suppress the answer 0, and ordinal words are worth 0.35 accuracy), Arabic speech recognition (isolated from visual reasoning by the parallel track design), and tie-breaking (calibration collapses ranks and a second judge separates them, while error correlation of 0.89 rules out cancellation). A post-submission study on degraded audio finds four transcript-level interventions null and measures a 9% answer-flip rate between identical runs at temperature 0, a noise floor for single-run ablations.\n- Anthology ID:\n- 2026.arabicnlp-sharedtasks.89\n- Volume:\n- [Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks](https://aclanthology.org/volumes/2026.arabicnlp-sharedtasks/)\n- Month:\n- October\n- Year:\n- 2026\n- Address:\n- Budapest, Hungary\n- Editors:\n- [Muhammad Abdul-Mageed](https://aclanthology.org/people/muhammad-abdul-mageed/) ,[Tamer Elsayed](https://aclanthology.org/people/tamer-elsayed/) ,[Houda Bouamor](https://aclanthology.org/people/houda-bouamor/unverified/) ,[Samhaa El-Beltagy](https://aclanthology.org/people/samhaa-r-el-beltagy/) ,[Hend Al-Khalifa](https://aclanthology.org/people/hend-al-khalifa/) ,[Ahmed Abdelali](https://aclanthology.org/people/ahmed-abdelali/unverified/) ,[AbdelRahim Elmadany](https://aclanthology.org/people/abdelrahim-elmadany/unverified/) ,[Abdellah El Mekki](https://aclanthology.org/people/abdellah-el-mekki/) ,[Kareem Darwish](https://aclanthology.org/people/kareem-darwish/) ,[Sakhar Alkhereyf](https://aclanthology.org/people/sakhar-alkhereyf/) ,[Hamzah Luqman](https://aclanthology.org/people/hamzah-luqman/) ,[Salam Khalifa](https://aclanthology.org/people/salam-khalifa/) ,[Samar Magdy](https://aclanthology.org/people/samar-magdy/unverified/) ,[Zaid Alyafeai](https://aclanthology.org/people/zaid-alyafeai/) ,[Injy Hamed](https://aclanthology.org/people/injy-hamed/unverified/) ,[Wajdi Zaghouani](https://aclanthology.org/people/wajdi-zaghouani/) ,[Imed Zitouni](https://aclanthology.org/people/imed-zitouni/unverified/)\n- Venue:\n- [ArabicNLP](https://aclanthology.org/venues/arabicnlp/)\n- SIG:\n- [SIGARAB](https://aclanthology.org/sigs/sigarab/)\n- Publisher:\n- Association for Computational Linguistics\n- Note:\n- Pages:\n- 711–720\n- Language:\n- URL:\n- [https://aclanthology.org/2026.arabicnlp-sharedtasks.89/](https://aclanthology.org/2026.arabicnlp-sharedtasks.89/)\n- DOI:\n- Cite (ACL):\n- Ahmed Ayman Ahmed Ezzat Mohamed. 2026. [Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation](https://aclanthology.org/2026.arabicnlp-sharedtasks.89/) . In*Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks* , pages 711–720, Budapest, Hungary. Association for Computational Linguistics.\n- Cite (Informal):\n- [Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation](https://aclanthology.org/2026.arabicnlp-sharedtasks.89/) (Mohamed, ArabicNLP 2026)\n- PDF:\n- [https://aclanthology.org/2026.arabicnlp-sharedtasks.89.pdf](https://aclanthology.org/2026.arabicnlp-sharedtasks.89.pdf)", "url": "https://wpnews.pro/news/ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken", "canonical_source": "https://aclanthology.org/2026.arabicnlp-sharedtasks.89/", "published_at": "2026-10-23 00:00:00+00:00", "updated_at": "2026-10-24 07:45:32.677309+00:00", "lang": "en", "topics": ["natural-language-processing", "computer-vision", "ai-research", "large-language-models"], "entities": ["Ahmed Ayman Ahmed Ezzat Mohamed", "ImageEval 2026", "CRAI-Bench", "ArabicNLP 2026", "Association for Computational Linguistics", "Modern Standard Arabic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken", "markdown": "https://wpnews.pro/news/ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken.md", "text": "https://wpnews.pro/news/ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken.txt", "jsonld": "https://wpnews.pro/news/ahmed-ayman-at-imageeval-2026-shared-tasks-measurement-first-systems-for-spoken.jsonld"}}