Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets

Abstract

Large Language Models (LLMs) are highly capable, state-of-the-art technologies and widely used as text classifiers for various NLP tasks, including sentiment analysis, topic classification, legal document analysis, etc. In this paper, we present a systematic analysis of the performance of LLMs as text classifiers using five German datasets from social media across 13 different tasks. We investigate zero- (ZSC) and few-shot classification (FSC) approaches with multiple LLMs and provide a comparative analysis with fine-tuned models based on Llama-3.2, EuroLLM, Teuken and BübleLM. We concentrate on investigating the limits of LLMs and on accurately describing our findings and overall challenges.

- Anthology ID:
- 2025.jlcl-2.1
- Volume:

• Journal for Language Technology and Computational Linguistics, Vol. 38 No. 2: LLM Fails - Failed experiments with generative AI and what we can learn from them
• Month:
• July
• Year:
• 2025

- Address:
- Editors:

• Ngoc Duyen Tanja Tu ,Annelen Brunner ,Christian Lang

- Venue:
- [JLCL](https://aclanthology.org/venues/jlcl/)
- SIG:
- Publisher:

• German Society for Computational Linguistics and Language Technology

- Note:
- Pages:

• 1–12

- Language:
- URL:
- [https://aclanthology.org/2025.jlcl-2.1/](https://aclanthology.org/2025.jlcl-2.1/)
- DOI:
- [10.21248/jlcl.38.2025.277](https://doi.org/10.21248/jlcl.38.2025.277)
- Cite (ACL):

• Elena Leitner and Georg Rehm. 2025. Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets .Journal for Language Technology and Computational Linguistics , 38(2):1–12.
• Cite (Informal):
• Exploring the Limits of LLMs for German Text Classification: Prompting and Fine-tuning Strategies Across Small and Medium-sized Datasets (Leitner & Rehm, JLCL 2025)

- PDF:
- [https://aclanthology.org/2025.jlcl-2.1.pdf](https://aclanthology.org/2025.jlcl-2.1.pdf)