{"slug": "stability-plasticity-balance-via-singular-vector-selection-in-llm-continual", "title": "Stability-Plasticity Balance via Singular-Vector Selection in LLM Continual Learning", "summary": "Researchers introduced SVC, a parameter-efficient continual-learning method that selectively updates singular-vector channels in LLMs to balance plasticity against catastrophic forgetting, according to arXiv paper 2610.11076v1. SVC estimates each channel's adaptation benefit from domain-specific data and its forgetting cost using a fixed public general-domain corpus as a history activation proxy, then selects trainable channels via knee-based cost screening, Pareto-front filtering, and Otsu thresholding. Across four LLM families and eight downstream tasks, SVC preserved pretrained capabilities better than existing PEFT baselines while maintaining strong downstream performance.", "body_md": "arXiv:2610.11076v1 Announce Type: new \nAbstract: Domain-specific continual adaptation of LLMs risks catastrophic forgetting, creating a fundamental tension between acquiring new capabilities and preserving those learned during pretraining. PEFT mitigates this problem by restricting the number of trainable parameters, but existing methods lack a principled unit for deciding where plasticity should be allocated and stability should be preserved. We identify the singular-vector channel as a natural unit for managing this trade-off. Each channel represents an input-output transformation, which can be updated to acquire new knowledge or fixed to preserve pretrained capabilities. Based on this perspective, we introduce SVC, a parameter-efficient continual-learning method that selectively updates Singular-Vector Channels. Before fine-tuning, SVC uses domain-specific data to estimate each channel's adaptation benefit and a fixed public general-domain corpus only as a history activation proxy for estimating forgetting cost. It then adaptively selects trainable channels based on these scores via knee-based cost screening, Pareto-front filtering, and Otsu thresholding. Experimental results across four LLM families and eight downstream tasks show that SVC better preserves pretrained capabilities while achieving strong downstream performance relative to existing PEFT baselines. Further analysis of channel scoring and selection demonstrates that selective plasticity at the singular-vector-channel level enables effective continual LLM adaptation.", "url": "https://wpnews.pro/news/stability-plasticity-balance-via-singular-vector-selection-in-llm-continual", "canonical_source": "https://www.machinebrief.com/news/stability-plasticity-balance-via-singular-vector-selection-i-x5y4", "published_at": "2026-10-09 04:00:00+00:00", "updated_at": "2026-10-09 04:47:05.606520+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["arXiv", "SVC", "PEFT"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stability-plasticity-balance-via-singular-vector-selection-in-llm-continual", "markdown": "https://wpnews.pro/news/stability-plasticity-balance-via-singular-vector-selection-in-llm-continual.md", "text": "https://wpnews.pro/news/stability-plasticity-balance-via-singular-vector-selection-in-llm-continual.txt", "jsonld": "https://wpnews.pro/news/stability-plasticity-balance-via-singular-vector-selection-in-llm-continual.jsonld"}}