sink_deflation

mentions 1 type Organization feed RSS

// recent coverage 1 mentions

20:05

2026-10-08

discuss.huggingface.co

large-language-models

Keeping the Attention Sink Exact in Activation-Whitened SVD Compression

A zero-cost reweighting technique that keeps the attention sink at token position 0 exact during activation-whitened SVD compression lowered the loss increase in 18 of 20 model-and-budget cells across…

// co-occurs with top 7 entities

SVD-LLM 1 KFAC-SVD 1 Qwen3-4B 1 Qwen3-32B 1 Qwen3-1.7B 1 Llama-3.1-8B 1 LEMS 1

// topics top 3 topics

large language models 1 machine learning 1 ai research 1