{"slug": "capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge", "title": "Capacity Allocation at the Source: Sparse Target Optimization for LLM Knowledge Editing", "summary": "A new arXiv paper (2611.12995v1) proposes an ℓ1 regularization method that prunes unnecessary components from the target hidden states used in large language model knowledge editing, reducing interference with existing knowledge. The authors establish a theoretical link showing that interference in parameter space is positively correlated with interference between target hidden states, and report that experiments across multiple LLMs and benchmark datasets verify the method's effectiveness. Code for the approach, called Sparse Target Optimization, is released at github.com/wintersdragon-c/STO.", "body_md": "arXiv:2611.12995v1 Announce Type: new \nAbstract: Knowledge editing has emerged as a promising approach for efficiently updating embedded knowledge in large language models (LLMs). It first computes an ideal target hidden state that steers the LLM toward the new output, and then edits the model parameters so that the original hidden state is mapped to this target. While most existing methods focus on the second stage of parameter editing in preserving other knowledge, we find that the upstream construction of the target hidden state also plays an important role in allocating the model's limited capacity space. We first establish a theoretical connection between interference in the parameter space and interference between target hidden states, showing that the two are positively correlated. Building on this insight, we propose a simple yet effective $\\ell_1$ regularization method that prunes away unnecessary ``tentacles'' of each knowledge vector, retaining only the essential components. With fewer extraneous tentacles, each edit affects less other knowledge, enabling better coexistence. Extensive experiments across multiple LLMs and benchmark datasets verify the effectiveness of our method. Code: https://github.com/wintersdragon-c/STO.", "url": "https://wpnews.pro/news/capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge", "canonical_source": "https://arxiv.org/abs/2611.12995", "published_at": "2026-11-11 05:00:00+00:00", "updated_at": "2026-11-11 05:19:33.382927+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-research"], "entities": ["arXiv", "Sparse Target Optimization", "STO", "github.com/wintersdragon-c/STO"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge", "markdown": "https://wpnews.pro/news/capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge.md", "text": "https://wpnews.pro/news/capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge.txt", "jsonld": "https://wpnews.pro/news/capacity-allocation-at-the-source-sparse-target-optimization-for-llm-knowledge.jsonld"}}