{"slug": "large-language-models-for-terminology-work-a-question-of-the-right-prompt", "title": "Large language models for terminology work: A question of the right prompt?", "summary": "A study by Barbara Heinisch published in the Journal for Language Technology and Computational Linguistics (Vol. 38 No. 2, pages 13–30, July 2025) found that ChatGPT, Perplexity and Microsoft CoPilot failed to reliably distinguish between the Austrian and German higher education systems and fabricated terms during terminology work, even when strategic prompting frameworks specified language variety and system context. Heinisch attributed the failures to the distribution of German-language training data, potential pivot translation via English, and a lack of transparency in LLM training, and noted output variability across identical prompts. The study concludes that domain-specific knowledge and LLM literacy are required to critically assess LLM-generated terminology, and that LLMs should be applied selectively rather than indiscriminately to terminology tasks.", "body_md": "##### Abstract\n\nText-generative large language models (LLMs) offer promising possibilities for terminology work, including term extraction, definition creation and assessment of concept relations. This study examines the performance of ChatGPT, Perplexity and Microsoft CoPilot for conducting terminology work in the field of the Austrian and British higher education systems using strategic prompting frameworks. Despite efforts to refine prompts by specifying language variety and system context, the LLM outputs failed to reliably differentiate between the Austrian and German systems and fabricated terms. Factors such as the distribution of German-language training data, potential pivot translation via English and the lack of transparency in LLM training further complicated evaluation. Additionally, output variability across identical prompts highlights the unpredictability of LLM-generated terminology. The study underscores the importance of human expertise in evaluating LLM outputs, as inconsistencies may undermine the reliability of terminology derived from such models. Without domain-specific knowledge (encompassing both subject-matter expertise and familiarity with terminology principles) as well as LLM literacy, users are unable to critically assess the quality of LLM outputs in terminological contexts. Rather than indiscriminately applying LLMs to all aspects of terminology work, it is crucial to assess their suitability for specific tasks.\n- Anthology ID:\n- 2025.jlcl-2.2\n- Volume:\n- [Journal for Language Technology and Computational Linguistics, Vol. 38 No. 2: LLM Fails - Failed experiments with generative AI and what we can learn from them](https://aclanthology.org/volumes/2025.jlcl-2/)\n- Month:\n- July\n- Year:\n- 2025\n- Address:\n- Editors:\n- [Ngoc Duyen Tanja Tu](https://aclanthology.org/people/ngoc-duyen-tanja-tu/unverified/) ,[Annelen Brunner](https://aclanthology.org/people/annelen-brunner/unverified/) ,[Christian Lang](https://aclanthology.org/people/christian-lang/unverified/)\n- Venue:\n- [JLCL](https://aclanthology.org/venues/jlcl/)\n- SIG:\n- Publisher:\n- German Society for Computational Linguistics and Language Technology\n- Note:\n- Pages:\n- 13–30\n- Language:\n- URL:\n- [https://aclanthology.org/2025.jlcl-2.2/](https://aclanthology.org/2025.jlcl-2.2/)\n- DOI:\n- [10.21248/jlcl.38.2025.280](https://doi.org/10.21248/jlcl.38.2025.280)\n- Cite (ACL):\n- Barbara Heinisch. 2025. [Large language models for terminology work: A question of the right prompt?](https://aclanthology.org/2025.jlcl-2.2/) .*Journal for Language Technology and Computational Linguistics* , 38(2):13–30.\n- Cite (Informal):\n- [Large language models for terminology work: A question of the right prompt?](https://aclanthology.org/2025.jlcl-2.2/) (Heinisch, JLCL 2025)\n- PDF:\n- [https://aclanthology.org/2025.jlcl-2.2.pdf](https://aclanthology.org/2025.jlcl-2.2.pdf)", "url": "https://wpnews.pro/news/large-language-models-for-terminology-work-a-question-of-the-right-prompt", "canonical_source": "https://aclanthology.org/2025.jlcl-2.2/", "published_at": "2026-10-14 00:00:00+00:00", "updated_at": "2026-10-16 12:47:35.827324+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "ai-research", "generative-ai"], "entities": ["Barbara Heinisch", "ChatGPT", "Perplexity", "Microsoft CoPilot", "Journal for Language Technology and Computational Linguistics", "German Society for Computational Linguistics and Language Technology"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/large-language-models-for-terminology-work-a-question-of-the-right-prompt", "markdown": "https://wpnews.pro/news/large-language-models-for-terminology-work-a-question-of-the-right-prompt.md", "text": "https://wpnews.pro/news/large-language-models-for-terminology-work-a-question-of-the-right-prompt.txt", "jsonld": "https://wpnews.pro/news/large-language-models-for-terminology-work-a-question-of-the-right-prompt.jsonld"}}