Zero-order Parameter-free Optimization for LMO-based Methods: Novel Approach for Efficient Fine-tuning
Researchers introduced AdaNAGED, a zero-order parameter-free optimization method for LMO-based fine-tuning of large language models, addressing memory overhead and sensitivity to hyperparameters. The method unifies gradi…