A new framework called LAARA (Layer-Aware Adaptive Rank Allocation) dynamically assigns adapter ranks per transformer layer using lightweight diagonal Fisher estimates, outperforming LoRA, AdaLoRA, Dy…
Researchers propose RSLoRA, a training-free and gradient-free rank allocator for Low-Rank Adaptation (LoRA) that uses activation-space geometry to assign rank capacity based on representational sensit…
Researchers introduced ReCoLoRA, a spectrum-aware recursive consolidation method for continual fine-tuning of large language models, which prevents task interference by re-decomposing the effective we…
Hugging Face's PEFT library challenges the dominance of LoRA in parameter-efficient fine-tuning, revealing that while LoRA accounts for over 98% of PEFT usage on the Hugging Face Hub, alternative meth…