CORA: Rethinking Efficient Fine-Tuning with Orthogonal Rotations
Researchers introduced CORA (Coherent Orthogonal Rotation Adaptation), a parameter-efficient fine-tuning method that uses orthogonal rotations and diagonal scaling to outperform existing approaches like LoRA with up to 8…