Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation

Ahmed Ayman Ahmed Ezzat Mohamed's system ranked first on the English track and second on Modern Standard Arabic in the ImageEval 2026 Subtask 1a spoken visual QA task, scoring 0.962 on English and 0.834 on MSA against organizer baselines of 0.664 and 0.398, according to the paper published in the Proceedings of the Fourth Arabic Natural Language Processing Conference (ArabicNLP 2026, pages 711–720, Budapest). On Task 2 (CRAI-Bench), Mohamed's judge ensemble reached a test Spearman of 0.673 versus 0.533 for a reproduction of the organizer baseline. The paper attributes the margins to answer-token bias, Arabic speech recognition isolated by the parallel track design, and tie-breaking via a second judge, and reports a post-submission study finding a 9% answer-flip rate between identical runs at temperature 0.

Abstract We describe our systems for both ImageEval 2026 shared tasks. On Subtask 1a spoken visual QA we reach 0.962 on English and 0.834 on Modern Standard Arabic against baselines of 0.664 and 0.398, ranking first on the English track and second on MSA, while on Task 2 CRAI-Bench our judge ensemble reaches a test Spearman of 0.673 against 0.533 for our reproduction of the organizer baseline. Three effects account for most of these margins: answer-token bias digit labels suppress the answer 0, and ordinal words are worth 0.35 accuracy , Arabic speech recognition isolated from visual reasoning by the parallel track design , and tie-breaking calibration collapses ranks and a second judge separates them, while error correlation of 0.89 rules out cancellation . A post-submission study on degraded audio finds four transcript-level interventions null and measures a 9% answer-flip rate between identical runs at temperature 0, a noise floor for single-run ablations. - Anthology ID: - 2026.arabicnlp-sharedtasks.89 - Volume: - Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks https://aclanthology.org/volumes/2026.arabicnlp-sharedtasks/ - Month: - October - Year: - 2026 - Address: - Budapest, Hungary - Editors: - Muhammad Abdul-Mageed https://aclanthology.org/people/muhammad-abdul-mageed/ , Tamer Elsayed https://aclanthology.org/people/tamer-elsayed/ , Houda Bouamor https://aclanthology.org/people/houda-bouamor/unverified/ , Samhaa El-Beltagy https://aclanthology.org/people/samhaa-r-el-beltagy/ , Hend Al-Khalifa https://aclanthology.org/people/hend-al-khalifa/ , Ahmed Abdelali https://aclanthology.org/people/ahmed-abdelali/unverified/ , AbdelRahim Elmadany https://aclanthology.org/people/abdelrahim-elmadany/unverified/ , Abdellah El Mekki https://aclanthology.org/people/abdellah-el-mekki/ , Kareem Darwish https://aclanthology.org/people/kareem-darwish/ , Sakhar Alkhereyf https://aclanthology.org/people/sakhar-alkhereyf/ , Hamzah Luqman https://aclanthology.org/people/hamzah-luqman/ , Salam Khalifa https://aclanthology.org/people/salam-khalifa/ , Samar Magdy https://aclanthology.org/people/samar-magdy/unverified/ , Zaid Alyafeai https://aclanthology.org/people/zaid-alyafeai/ , Injy Hamed https://aclanthology.org/people/injy-hamed/unverified/ , Wajdi Zaghouani https://aclanthology.org/people/wajdi-zaghouani/ , Imed Zitouni https://aclanthology.org/people/imed-zitouni/unverified/ - Venue: - ArabicNLP https://aclanthology.org/venues/arabicnlp/ - SIG: - SIGARAB https://aclanthology.org/sigs/sigarab/ - Publisher: - Association for Computational Linguistics - Note: - Pages: - 711–720 - Language: - URL: - https://aclanthology.org/2026.arabicnlp-sharedtasks.89/ https://aclanthology.org/2026.arabicnlp-sharedtasks.89/ - DOI: - Cite ACL : - Ahmed Ayman Ahmed Ezzat Mohamed. 2026. Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation https://aclanthology.org/2026.arabicnlp-sharedtasks.89/ . In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks , pages 711–720, Budapest, Hungary. Association for Computational Linguistics. - Cite Informal : - Ahmed Ayman at ImageEval 2026 Shared Tasks: Measurement-First Systems for Spoken VQA and Cultural Accuracy Evaluation https://aclanthology.org/2026.arabicnlp-sharedtasks.89/ Mohamed, ArabicNLP 2026 - PDF: - https://aclanthology.org/2026.arabicnlp-sharedtasks.89.pdf https://aclanthology.org/2026.arabicnlp-sharedtasks.89.pdf