{"slug": "energy-structured-low-rank-adaptation-for-continual-learning", "title": "Energy-Structured Low-Rank Adaptation for Continual Learning", "summary": "Researchers introduced E²-LoRA, a method that concentrates knowledge into leading ranks of parameter updates to prevent energy diffusion and free capacity for new tasks in continual learning. The approach, which includes a dynamic rank allocation strategy balancing stability and plasticity, achieved state-of-the-art performance across multiple benchmarks.", "body_md": "arXiv:2605.27482v1 Announce Type: new\nAbstract: While orthogonal subspace methods try to mitigate task interference in Continual Learning (CL), they often suffer from energy diffusion across the basis, hindering knowledge compaction and exhausting capacity for future tasks. We observe that output feature drift induced by parameter updates is inherently low-rank, and theoretically prove that preserving parameters along the principal directions of this drift minimizes the output reconstruction error. Motivated by this, we propose \\textbf{E}nergy-Concentrated and \\textbf{E}nergy-Ordered \\textbf{Lo}w-\\textbf{R}ank \\textbf{A}daptation (E$^2$-LoRA). By explicitly ordering and concentrating knowledge into leading ranks, E$^2$-LoRA frees capacity for subsequent tasks. Furthermore, we design a dynamic rank allocation strategy to balance stability and plasticity by jointly optimizing energy retention and model plasticity. Extensive experiments across multiple benchmarks demonstrate that E$^2$-LoRA achieves state-of-the-art performance.", "url": "https://wpnews.pro/news/energy-structured-low-rank-adaptation-for-continual-learning", "canonical_source": "https://arxiv.org/abs/2605.27482", "published_at": "2026-05-28 04:00:00+00:00", "updated_at": "2026-05-28 04:29:14.657465+00:00", "lang": "en", "topics": ["machine-learning", "neural-networks", "artificial-intelligence", "ai-research"], "entities": ["E$^2$-LoRA"], "alternates": {"html": "https://wpnews.pro/news/energy-structured-low-rank-adaptation-for-continual-learning", "markdown": "https://wpnews.pro/news/energy-structured-low-rank-adaptation-for-continual-learning.md", "text": "https://wpnews.pro/news/energy-structured-low-rank-adaptation-for-continual-learning.txt", "jsonld": "https://wpnews.pro/news/energy-structured-low-rank-adaptation-for-continual-learning.jsonld"}}