{"slug": "lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and", "title": "LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment", "summary": "Researchers introduced LARA (Lightweight Additive Residual Adaptation), a method that adapts frozen AI models by adding low-rank corrections to the residual stream instead of modifying weights, matching LoRA's performance on code fine-tuning and DPO at equal parameter counts. LARA enables smooth interpolation between base and adapted behavior via a scale parameter and supports hosting multiple behaviors on a single model, demonstrated by placing seven behaviors on one frozen 1.5B model with roughly 33 MB of overhead.", "body_md": "arXiv:2607.28669v1 Announce Type: new\nAbstract: We present LARA (Lightweight Additive Residual Adaptation), a method for efficient adaptation that operates in the residual stream of a frozen model rather than in its weights. Where LoRA adds an update of low rank to weight matrices, LARA reads the hidden state at a small set of layers and adds a correction of low rank back to the residual stream, leaving all base weights untouched. On a code fine-tuning task and on preference optimization (DPO), LARA matches LoRA at equal parameter counts. Because adaptation is a frozen base plus a residual, LARA exposes a scale {\\gamma}, applied at inference, that interpolates smoothly between base and adapted behavior, a form of graded control that adaptation in weight space does not offer. Finally, because each behavior is a small residual module over a shared frozen base, many behaviors can be held resident at once and routed automatically per token. We place seven behaviors, six fine-tuned and one optimized for preference, on one frozen 1.5B model for roughly 33 MB of overhead, against one full model for each behavior. Because the base is untouched, behaviors are trained separately and selected per token rather than loaded on demand, which suits hosting many behaviors, and adding new ones, on a single model on a device.", "url": "https://wpnews.pro/news/lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and", "canonical_source": "https://arxiv.org/abs/2607.28669", "published_at": "2026-08-03 04:00:00+00:00", "updated_at": "2026-08-03 04:02:21.801315+00:00", "lang": "en", "topics": ["machine-learning", "artificial-intelligence"], "entities": ["LARA", "LoRA"], "alternates": {"html": "https://wpnews.pro/news/lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and", "markdown": "https://wpnews.pro/news/lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and.md", "text": "https://wpnews.pro/news/lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and.txt", "jsonld": "https://wpnews.pro/news/lara-lightweight-adapters-in-the-residual-stream-for-composable-adaptation-and.jsonld"}}