{"slug": "spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral", "title": "SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization", "summary": "A method called SpectralShift extends the effective context window of Gated DeltaNet, a linear attention architecture used in place of softmax attention for long-context modeling, by reparameterizing the model's spectral properties. The approach addresses a gap in existing context extension techniques, which apply continued pretraining directly to linear attention layers without modifying them and therefore overlook those layers' spectral properties.", "body_md": "Recently, linear attention layers have been increasingly adopted to replace softmax attention at scale for long-context modeling. However, existing context extension approaches typically apply continued pretraining directly without modifying these layers, overlooking the spectral properties of linea", "url": "https://wpnews.pro/news/spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral", "canonical_source": "https://aiflash.com/news/121161/", "published_at": "2026-09-17 05:34:09+00:00", "updated_at": "2026-09-17 05:53:59.739028+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "ai-research", "natural-language-processing"], "entities": ["SpectralShift", "Gated DeltaNet"], "alternates": {"html": "https://wpnews.pro/news/spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral", "markdown": "https://wpnews.pro/news/spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral.md", "text": "https://wpnews.pro/news/spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral.txt", "jsonld": "https://wpnews.pro/news/spectralshift-effective-context-window-extension-of-gated-deltanet-via-spectral.jsonld"}}