{"slug": "hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language", "title": "HyperRepair: Recovering Forgotten General Capabilities of Fine-Tuned Language Models", "summary": "A framework called HyperRepair recovers 93.4% of base language models' average general benchmark score while retaining 99.5% of fine-tuned models' average domain score, according to an arXiv paper (2610.41147v1). HyperRepair learns a hypernetwork from repairs trained on source models and uses activation fingerprints to generate a prompt-conditioned Repairing Side Net that applies corrections selectively, enabling recovery on models from unseen adaptation domains without target-domain examples or further optimization. The authors evaluated HyperRepair across two model families, 12 LoRA adaptation datasets and 15 general benchmarks, addressing the trade-off between general capability and domain specialization at inference-only cost.", "body_md": "arXiv:2610.41147v1 Announce Type: new \nAbstract: Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to preserve both. We introduce HyperRepair, a framework for recovering general capabilities while retaining domain specialization at only inference cost. To make recovery generalizable across models sharing a pretrained backbone, HyperRepair learns a hypernetwork from repairs trained on source models. The hypernetwork uses activation fingerprints that summarize changes from the pretrained backbone to generate a tailored Repairing Side Net (RSN), whose prompt-conditioned gate applies corrections selectively. This enables recovery on models from unseen adaptation domains without target-domain examples or additional optimization. We evaluate HyperRpair across two model families, 12 LoRA adaptation datasets, and 15 general benchmarks. On the default split, HyperRepair achieves 93.4% of the base models' average general score while retaining 99.5% of the fine-tuned models' average domain score.", "url": "https://wpnews.pro/news/hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language", "canonical_source": "https://arxiv.org/abs/2610.41147", "published_at": "2026-11-02 05:00:00+00:00", "updated_at": "2026-11-02 05:20:00.629775+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research", "natural-language-processing"], "entities": ["HyperRepair", "Repairing Side Net", "LoRA", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language", "markdown": "https://wpnews.pro/news/hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language.md", "text": "https://wpnews.pro/news/hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language.txt", "jsonld": "https://wpnews.pro/news/hyperrepair-recovering-forgotten-general-capabilities-of-fine-tuned-language.jsonld"}}