# A Study of Errors in the Output of Large Language Models for Domain-Specific Few-Shot Named Entity Recognition

> Source: <https://aclanthology.org/2025.jlcl-2.3/>
> Published: 2026-10-14 00:00:00+00:00

##### Abstract

This paper proposes an error classification framework for a comprehensive analysis of the output that large language models (LLMs) generate in a few-shot named entity recognition (NER) task in a specialised domain. The framework should be seen as an exploratory analysis complementary to established performance metrics for NER classifiers, such as F1 score, as it accounts for outcomes possible in a few-shot, LLM-based NER task. By categorising and assessing incorrect named entity predictions quantitatively, the paper shows how the proposed error classification could support a deeper cross-model and cross-prompt performance comparison, alongside a roadmap for a guided qualitative error analysis.
- Anthology ID:
- 2025.jlcl-2.3
- Volume:
- [Journal for Language Technology and Computational Linguistics, Vol. 38 No. 2: LLM Fails - Failed experiments with generative AI and what we can learn from them](https://aclanthology.org/volumes/2025.jlcl-2/)
- Month:
- July
- Year:
- 2025
- Address:
- Editors:
- [Ngoc Duyen Tanja Tu](https://aclanthology.org/people/ngoc-duyen-tanja-tu/unverified/) ,[Annelen Brunner](https://aclanthology.org/people/annelen-brunner/unverified/) ,[Christian Lang](https://aclanthology.org/people/christian-lang/unverified/)
- Venue:
- [JLCL](https://aclanthology.org/venues/jlcl/)
- SIG:
- Publisher:
- German Society for Computational Linguistics and Language Technology
- Note:
- Pages:
- 31–42
- Language:
- URL:
- [https://aclanthology.org/2025.jlcl-2.3/](https://aclanthology.org/2025.jlcl-2.3/)
- DOI:
- [10.21248/jlcl.38.2025.281](https://doi.org/10.21248/jlcl.38.2025.281)
- Cite (ACL):
- Elena Volkanovska. 2025. [A Study of Errors in the Output of Large Language Models for Domain-Specific Few-Shot Named Entity Recognition](https://aclanthology.org/2025.jlcl-2.3/) .*Journal for Language Technology and Computational Linguistics* , 38(2):31–42.
- Cite (Informal):
- [A Study of Errors in the Output of Large Language Models for Domain-Specific Few-Shot Named Entity Recognition](https://aclanthology.org/2025.jlcl-2.3/) (Volkanovska, JLCL 2025)
- PDF:
- [https://aclanthology.org/2025.jlcl-2.3.pdf](https://aclanthology.org/2025.jlcl-2.3.pdf)