Abstract
Data-sovereignty rules forbid cloud-hosted AI in many high-security environments, leaving compact on-premise models as the only path to AI-assisted cybersecurity. We fine-tune three small open-source models, Gemma 2 2B, Phi-3.5 3.8B, Llama 3.1 8B, on ~147,600 synthetic cybersecurity QA pairs using QLoRA on V100 GPUs. Under strict MCQ evaluation Gemma 2 gains +9.3 pp, Phi +4.0 pp, and Llama drops −24.0 pp. We term this the educator effect: models trained on pedagogical data internalize explanatory behavior at the expense of format compliance. Severity appears to scale with capacity, though capacity is confounded with architecture and learning rate. A controlled ablation shows randomized ordering outperforms curriculum, without significance testing on the 75-question exam.- Anthology ID:
- 2026.nlpaics-1.6
- Volume:
- June
- Year:
- 2026
- Address:
- Alicante, Spain
- Editors: Ruslan Mitkov,Rafael Muñoz,Elena Lloret,Tharindu Ranasinghe,Ernesto L. Estevanell-Valladares,Salima Lamsiyah,Andrés Montoyo,Saad Ezzini- Venue:
[NLPAICS](/venues/nlpaics/)- SIG:
- Publisher:
- Department of Languages and Information Systems, University of Alicante
- Note:
- Pages:
- 55–63
- Language:
- URL:
[https://aclanthology.org/2026.nlpaics-1.6/](https://aclanthology.org/2026.nlpaics-1.6/)- DOI:
- Cite (ACL):
- Ozkan Kilic, Raja Soundaramourty, and Ramu Chenchaiah. 2026. Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect. InProceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security, pages 55–63, Alicante, Spain. Department of Languages and Information Systems, University of Alicante. - Cite (Informal):
[Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect](https://aclanthology.org/2026.nlpaics-1.6/)(Kilic et al., NLPAICS 2026)- PDF:
[https://aclanthology.org/2026.nlpaics-1.6.pdf](https://aclanthology.org/2026.nlpaics-1.6.pdf)