Advancing SVD-based LLM Compression via Layer-Wise Error Model Search

A new research approach improves low-rank SVD-based LLM compression by replacing global rank allocation with layer-wise error model search, addressing what the authors identify as two key limitations in existing methods: uncalibrated error proxies that fail to capture complex error propagation, and decomposition quality issues. The work targets reducing the computational costs of large language models through low-rank singular value decomposition.

Low-rank SVD-based compression offers a powerful strategy to reduce the computational costs of LLMs. However, existing methods face two key limitations: i global rank allocation, where uncalibrated error proxies fail to capture complex error propagation, and ii decomposition quality, where Fishe