Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets

Elena Leitner and Georg Rehm published a systematic analysis in the Journal for Language Technology and Computational Linguistics (Vol. 38 No. 2, July 2025) evaluating large language models as German text classifiers across five social media datasets and 13 tasks, comparing zero-shot and few-shot prompting against fine-tuned models based on Llama-3.2, EuroLLM, Teuken and BübleLM. The paper, pages 1–12 with DOI 10.21248/jlcl.38.2025.277, focuses on the limits of LLMs and the challenges encountered in small and medium-sized dataset settings.

Abstract Large Language Models LLMs are highly capable, state-of-the-art technologies and widely used as text classifiers for various NLP tasks, including sentiment analysis, topic classification, legal document analysis, etc. In this paper, we present a systematic analysis of the performance of LLMs as text classifiers using five German datasets from social media across 13 different tasks. We investigate zero- ZSC and few-shot classification FSC approaches with multiple LLMs and provide a comparative analysis with fine-tuned models based on Llama-3.2, EuroLLM, Teuken and BübleLM. We concentrate on investigating the limits of LLMs and on accurately describing our findings and overall challenges. - Anthology ID: - 2025.jlcl-2.1 - Volume: - Journal for Language Technology and Computational Linguistics, Vol. 38 No. 2: LLM Fails - Failed experiments with generative AI and what we can learn from them https://aclanthology.org/volumes/2025.jlcl-2/ - Month: - July - Year: - 2025 - Address: - Editors: - Ngoc Duyen Tanja Tu https://aclanthology.org/people/ngoc-duyen-tanja-tu/unverified/ , Annelen Brunner https://aclanthology.org/people/annelen-brunner/unverified/ , Christian Lang https://aclanthology.org/people/christian-lang/unverified/ - Venue: - JLCL https://aclanthology.org/venues/jlcl/ - SIG: - Publisher: - German Society for Computational Linguistics and Language Technology - Note: - Pages: - 1–12 - Language: - URL: - https://aclanthology.org/2025.jlcl-2.1/ https://aclanthology.org/2025.jlcl-2.1/ - DOI: - 10.21248/jlcl.38.2025.277 https://doi.org/10.21248/jlcl.38.2025.277 - Cite ACL : - Elena Leitner and Georg Rehm. 2025. Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets https://aclanthology.org/2025.jlcl-2.1/ . Journal for Language Technology and Computational Linguistics , 38 2 :1–12. - Cite Informal : - Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets https://aclanthology.org/2025.jlcl-2.1/ Leitner & Rehm, JLCL 2025 - PDF: - https://aclanthology.org/2025.jlcl-2.1.pdf https://aclanthology.org/2025.jlcl-2.1.pdf