{"slug": "abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function", "title": "Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling", "summary": "Abjad's submission to the AISA-ArabicFC shared task Track B reached 96.15% function-name accuracy but only 40.6% argument exact-match on the 545-example development set, producing an official overall_b score of 0.6665, below the organizers' reported baseline of 0.739, according to the paper by Randa Lakab, Shahad Abdulaziz Abdullah Alsultan and Abrar Almutiri in the Proceedings of the Fourth Arabic Natural Language Processing Conference. Rather than fine-tuning, the team built an inference and evaluation pipeline around the organizers' AISA-AR-FunctionCall-Think baseline model, diagnosing and fixing two pipeline-level bugs that were silently depressing scores. The authors' error analysis found failures concentrate almost entirely in argument value formatting — currency codes, ISO language codes and transliteration — rather than tool selection or task comprehension, pointing to targeted fine-tuning or post-processing for structured-value normalization as a promising direction.", "body_md": "## [Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling](https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf)\n\n[Randa Lakab](https://aclanthology.org/people/randa-lakab/unverified/),\n[Shahad Abdulaziz Abdullah Alsultan](https://aclanthology.org/people/shahad-abdulaziz-abdullah-alsultan/unverified/),\n[Abrar Almutiri](https://aclanthology.org/people/abrar-almutiri/unverified/)\n\n##### Abstract\n\nWe describe Abjad’s submission to the AISA-ArabicFC shared task (Track B), which targets Arabic function calling with explicit natural-language reasoning prior to the structured call. Rather than fine-tuning, our system builds a robust inference and evaluation pipeline around the organizers’ baseline model, AISA-AR-FunctionCall-Think, diagnosing and fixing two pipeline-level bugs that were silently depressing scores. On the 545-example development set, our system reaches a function-name accuracy of 96.15% but an argument exact-match of only 40.6%, yielding an official Track B score (overall_b) of 0.6665, below the organizer-reported baseline of 0.739. We present a fine-grained error analysis showing that failures concentrate almost entirely in argument value formatting, including currency codes, ISO language codes, and transliteration, rather than in tool selection or task comprehension. These findings suggest that targeted fine-tuning or post-processing for structured-value normalization is a promising direction for improving Arabic function-calling performance. abjad aisa arabicfc.pdf\n- Anthology ID:\n- 2026.arabicnlp-sharedtasks.204\n- Volume:\n- [Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks](https://aclanthology.org/volumes/2026.arabicnlp-sharedtasks/)\n- Month:\n- October\n- Year:\n- 2026\n- Address:\n- Budapest, Hungary\n- Editors:\n- [Muhammad Abdul-Mageed](https://aclanthology.org/people/muhammad-abdul-mageed/) ,[Tamer Elsayed](https://aclanthology.org/people/tamer-elsayed/) ,[Houda Bouamor](https://aclanthology.org/people/houda-bouamor/unverified/) ,[Samhaa El-Beltagy](https://aclanthology.org/people/samhaa-r-el-beltagy/) ,[Hend Al-Khalifa](https://aclanthology.org/people/hend-al-khalifa/) ,[Ahmed Abdelali](https://aclanthology.org/people/ahmed-abdelali/unverified/) ,[AbdelRahim Elmadany](https://aclanthology.org/people/abdelrahim-elmadany/unverified/) ,[Abdellah El Mekki](https://aclanthology.org/people/abdellah-el-mekki/) ,[Kareem Darwish](https://aclanthology.org/people/kareem-darwish/) ,[Sakhar Alkhereyf](https://aclanthology.org/people/sakhar-alkhereyf/) ,[Hamzah Luqman](https://aclanthology.org/people/hamzah-luqman/) ,[Salam Khalifa](https://aclanthology.org/people/salam-khalifa/) ,[Samar Magdy](https://aclanthology.org/people/samar-magdy/unverified/) ,[Zaid Alyafeai](https://aclanthology.org/people/zaid-alyafeai/) ,[Injy Hamed](https://aclanthology.org/people/injy-hamed/unverified/) ,[Wajdi Zaghouani](https://aclanthology.org/people/wajdi-zaghouani/) ,[Imed Zitouni](https://aclanthology.org/people/imed-zitouni/unverified/)\n- Venue:\n- [ArabicNLP](https://aclanthology.org/venues/arabicnlp/)\n- SIG:\n- [SIGARAB](https://aclanthology.org/sigs/sigarab/)\n- Publisher:\n- Association for Computational Linguistics\n- Note:\n- Pages:\n- 1585–1591\n- Language:\n- URL:\n- [https://aclanthology.org/2026.arabicnlp-sharedtasks.204/](https://aclanthology.org/2026.arabicnlp-sharedtasks.204/)\n- DOI:\n- Cite (ACL):\n- Randa Lakab, Shahad Abdulaziz Abdullah Alsultan, and Abrar Almutiri. 2026. [Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling](https://aclanthology.org/2026.arabicnlp-sharedtasks.204/) . In*Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks* , pages 1585–1591, Budapest, Hungary. Association for Computational Linguistics.\n- Cite (Informal):\n- [Abjad at AISA-ArabicFC: Error Analysis and Prompt Engineering for Arabic Function Calling](https://aclanthology.org/2026.arabicnlp-sharedtasks.204/) (Lakab et al., ArabicNLP 2026)\n- PDF:\n- [https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf](https://aclanthology.org/2026.arabicnlp-sharedtasks.204.pdf)", "url": "https://wpnews.pro/news/abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function", "canonical_source": "https://aclanthology.org/2026.arabicnlp-sharedtasks.204/", "published_at": "2026-10-23 00:00:00+00:00", "updated_at": "2026-10-24 07:45:24.668318+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "ai-research", "ai-agents"], "entities": ["Abjad", "AISA-ArabicFC", "Randa Lakab", "Shahad Abdulaziz Abdullah Alsultan", "Abrar Almutiri", "AISA-AR-FunctionCall-Think", "Association for Computational Linguistics", "ArabicNLP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function", "markdown": "https://wpnews.pro/news/abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function.md", "text": "https://wpnews.pro/news/abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function.txt", "jsonld": "https://wpnews.pro/news/abjad-at-aisa-arabicfc-error-analysis-and-prompt-engineering-for-arabic-function.jsonld"}}