In a head-to-head test, standard LoRA outperformed Weight-decomposed Low-Rank Adaptation (DoRA) by 2.1% in accuracy while taking 15% less training time per epoch, according to a developer's experiment…
Researchers introduced CORA (Coherent Orthogonal Rotation Adaptation), a parameter-efficient fine-tuning method that uses orthogonal rotations and diagonal scaling to outperform existing approaches li…
Researchers introduced ReCoLoRA, a spectrum-aware recursive consolidation method for continual fine-tuning of large language models, which prevents task interference by re-decomposing the effective we…
Hugging Face's PEFT library challenges the dominance of LoRA in parameter-efficient fine-tuning, revealing that while LoRA accounts for over 98% of PEFT usage on the Hugging Face Hub, alternative meth…
Guide for fine-tuning NVIDIA's Cosmos Predict 2.5 world model using LoRA and DoRA techniques to generate synthetic robot manipulation videos. The approach freezes the base model's 2 billion parameters…