{"slug": "rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity", "title": "RSLoRA: Training-free Rank Allocation for LoRA via Representational Sensitivity Probing", "summary": "Researchers propose RSLoRA, a training-free rank allocation method for LoRA that uses activation-space geometry to assign ranks based on representational sensitivity. The method outperforms existing allocators like AdaLoRA and GoRA on benchmarks, eliminating the need for iterative training-time adjustments.", "body_md": "arXiv:2607.09757v1 Announce Type: new\nAbstract: Low-Rank Adaptation (LoRA) has become a cornerstone of parameter-efficient fine-tuning (PEFT); however, the conventional practice of uniform rank assignment ignores the functional heterogeneity of neural layers. Existing rank allocation methods typically struggle with a trade-off between computational intensity and heuristic simplicity: training-based methods suffer from prohibitive overhead, while pre-allocation methods fail to capture the dynamic task-specific representation manifold. In this paper, we propose RSLoRA (Representational Sensitivity LoRA), a training-free and gradient-free rank allocator driven by activation-space geometry. We identify a \"sensitivity regime shift\" across layers, observing that static weight analysis and local gradients are insufficient to reflect how updates reshape a model's internal representations. To address this, RSLoRA introduces a virtual representational probing mechanism. By simulating adaptation through structured low-rank noise and measuring the resulting manifold displacement by using Effective Rank and Frechet Distance, we identify high-sensitivity modules that require higher rank capacity. Our framework effectively bridges the gap between expert-crafted heuristics and actual representational impact. Extensive evaluations demonstrate that RSLoRA consistently outperforms state-of-the-art allocators (e.g., AdaLoRA, GoRA) across mainstream benchmarks. By eliminating the need for iterative training-time adjustments and backward gradients, RSLoRA provides a highly efficient, robust, and representation-aware solution for large-scale model adaptation.", "url": "https://wpnews.pro/news/rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity", "canonical_source": "https://arxiv.org/abs/2607.09757", "published_at": "2026-07-14 04:00:00+00:00", "updated_at": "2026-07-14 04:03:18.723394+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research"], "entities": ["RSLoRA", "LoRA", "AdaLoRA", "GoRA"], "alternates": {"html": "https://wpnews.pro/news/rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity", "markdown": "https://wpnews.pro/news/rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity.md", "text": "https://wpnews.pro/news/rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity.txt", "jsonld": "https://wpnews.pro/news/rslora-training-free-rank-allocation-for-lora-via-representational-sensitivity.jsonld"}}