Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling

Abjad's submission to the AISA-ArabicFC shared task Track B reached 96.15% function-name accuracy but only 40.6% argument exact-match on the 545-example development set, producing an official overall_b score of 0.6665, below the organizers' reported baseline of 0.739, according to the paper by Randa Lakab, Shahad Abdulaziz Abdullah Alsultan and Abrar Almutiri in the Proceedings of the Fourth Arabic Natural Language Processing Conference. Rather than fine-tuning, the team built an inference and evaluation pipeline around the organizers' AISA-AR-FunctionCall-Think baseline model, diagnosing and fixing two pipeline-level bugs that were silently depressing scores. The authors' error analysis found failures concentrate almost entirely in argument value formatting — currency codes, ISO language codes and transliteration — rather than tool selection or task comprehension, pointing to targeted fine-tuning or post-processing for structured-value normalization as a promising direction.

Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf Randa Lakab https://aclanthology.org/people/randa-lakab/unverified/ , Shahad Abdulaziz Abdullah Alsultan https://aclanthology.org/people/shahad-abdulaziz-abdullah-alsultan/unverified/ , Abrar Almutiri https://aclanthology.org/people/abrar-almutiri/unverified/ Abstract We describe Abjad’s submission to the AISA-ArabicFC shared task Track B , which targets Arabic function calling with explicit natural-language reasoning prior to the structured call. Rather than fine-tuning, our system builds a robust inference and evaluation pipeline around the organizers’ baseline model, AISA-AR-FunctionCall-Think, diagnosing and fixing two pipeline-level bugs that were silently depressing scores. On the 545-example development set, our system reaches a function-name accuracy of 96.15% but an argument exact-match of only 40.6%, yielding an official Track B score overall b of 0.6665, below the organizer-reported baseline of 0.739. We present a fine-grained error analysis showing that failures concentrate almost entirely in argument value formatting, including currency codes, ISO language codes, and transliteration, rather than in tool selection or task comprehension. These findings suggest that targeted fine-tuning or post-processing for structured-value normalization is a promising direction for improving Arabic function-calling performance. abjad aisa arabicfc.pdf - Anthology ID: - 2026.arabicnlp-sharedtasks.204 - Volume: - Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks https://aclanthology.org/volumes/2026.arabicnlp-sharedtasks/ - Month: - October - Year: - 2026 - Address: - Budapest, Hungary - Editors: - Muhammad Abdul-Mageed https://aclanthology.org/people/muhammad-abdul-mageed/ , Tamer Elsayed https://aclanthology.org/people/tamer-elsayed/ , Houda Bouamor https://aclanthology.org/people/houda-bouamor/unverified/ , Samhaa El-Beltagy https://aclanthology.org/people/samhaa-r-el-beltagy/ , Hend Al-Khalifa https://aclanthology.org/people/hend-al-khalifa/ , Ahmed Abdelali https://aclanthology.org/people/ahmed-abdelali/unverified/ , AbdelRahim Elmadany https://aclanthology.org/people/abdelrahim-elmadany/unverified/ , Abdellah El Mekki https://aclanthology.org/people/abdellah-el-mekki/ , Kareem Darwish https://aclanthology.org/people/kareem-darwish/ , Sakhar Alkhereyf https://aclanthology.org/people/sakhar-alkhereyf/ , Hamzah Luqman https://aclanthology.org/people/hamzah-luqman/ , Salam Khalifa https://aclanthology.org/people/salam-khalifa/ , Samar Magdy https://aclanthology.org/people/samar-magdy/unverified/ , Zaid Alyafeai https://aclanthology.org/people/zaid-alyafeai/ , Injy Hamed https://aclanthology.org/people/injy-hamed/unverified/ , Wajdi Zaghouani https://aclanthology.org/people/wajdi-zaghouani/ , Imed Zitouni https://aclanthology.org/people/imed-zitouni/unverified/ - Venue: - ArabicNLP https://aclanthology.org/venues/arabicnlp/ - SIG: - SIGARAB https://aclanthology.org/sigs/sigarab/ - Publisher: - Association for Computational Linguistics - Note: - Pages: - 1585–1591 - Language: - URL: - https://aclanthology.org/2026.arabicnlp-sharedtasks.204/ https://aclanthology.org/2026.arabicnlp-sharedtasks.204/ - DOI: - Cite ACL : - Randa Lakab, Shahad Abdulaziz Abdullah Alsultan, and Abrar Almutiri. 2026. Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling https://aclanthology.org/2026.arabicnlp-sharedtasks.204/ . In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks , pages 1585–1591, Budapest, Hungary. Association for Computational Linguistics. - Cite Informal : - Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling https://aclanthology.org/2026.arabicnlp-sharedtasks.204/ Lakab et al., ArabicNLP 2026 - PDF: - https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf