A Survey on Memory-Efficient Fine-Tuning for Large Language Models

Yeachan Kim, Mingyu Lee, and SangKeun Lee published "A Survey on Memory-Efficient Fine-Tuning for Large Language Models" in Transactions of the Association for Computational Linguistics, Volume 14, pages 960–982, in 2026. The survey presents what the authors describe as the first systematic review of memory-efficient fine-tuning (MEFT) for LLMs, categorizing existing approaches by optimization environment (the model itself and systems) and further classifying model-based approaches by optimization target. The paper also discusses evaluation strategies for MEFT methods and provides empirical analyses, aiming to serve as a practical guide for developing MEFT methods.

Abstract Fine-tuning large language models LLMs is a crucial process to align them with human intentions, yet this process remains memoryintensive, varying across tasks and model architectures. These huge and variable memory costs complicate scaling and deployment of LLMs, especially on limited hardware. However, existing surveys on memory efficiency are often either superficial or too narrow in scope, typically focusing on specific subfields. To address this gap, this survey presents the first systematic review of memory-efficient fine-tuning MEFT tailored for LLMs. To structure the research landscape, we first categorize existing approaches by their optimization environments i.e., model itself and systems and further classify model-based approaches by their specific optimization targets. We also discuss evaluation strategies for assessing MEFT methods and provide empirical analyses. By highlighting challenges and future directions based on current methods, this survey aims to serve as a practical guide for developing MEFT methods. - Anthology ID: - 2026.tacl-1.43 - Volume: - Transactions of the Association for Computational Linguistics, Volume 14 https://aclanthology.org/volumes/2026.tacl-1/ - Month: - Year: - 2026 - Address: - Cambridge, MA - Venue: - TACL https://aclanthology.org/venues/tacl/ - SIG: - Publisher: - MIT Press - Note: - Pages: - 960–982 - Language: - URL: - https://aclanthology.org/2026.tacl-1.43/ https://aclanthology.org/2026.tacl-1.43/ - DOI: - 10.1162/tacl.a.692 https://doi.org/10.1162/tacl.a.692 - Cite ACL : - Yeachan Kim, Mingyu Lee, and SangKeun Lee. 2026. A Survey on Memory-Efficient Fine-Tuning for Large Language Models https://aclanthology.org/2026.tacl-1.43/ . Transactions of the Association for Computational Linguistics , 14:960–982. - Cite Informal : - A Survey on Memory-Efficient Fine-Tuning for Large Language Models https://aclanthology.org/2026.tacl-1.43/ Kim et al., TACL 2026 - PDF: - https://aclanthology.org/2026.tacl-1.43.pdf https://aclanthology.org/2026.tacl-1.43.pdf