Building a Nested LADDER for LLM Unstructured Post-Training Pruning

A new arXiv paper (2610.28773v1) introduces LADDER (Layer-Wise Allocation via Damage-Driven Evaluation and Refinement), a post-training pruning method for large language models that allocates sparsity incrementally, committing at each step the layer update causing the least end-to-end degradation per removed parameter and producing exactly nested masks across multiple budgets. Evaluated on five OPT, Llama and Qwen models across 10-90% sparsity with SparseGPT as the candidate-generation backend, LADDER was comparable or slightly weaker than uniform and adaptive allocation baselines at benign sparsity but became substantially more robust from 70% sparsity onward, an advantage that largely persisted after posterior distillation in four of five models and improved retention of aggregate zero-shot accuracy.

arXiv:2610.28773v1 Announce Type: new Abstract: Non-uniform allocation can improve post-training pruning of large language models, yet quality can still degrade sharply at high sparsity. To control degradation as sparsity increases, we introduce LADDER Layer-Wise Allocation via Damage-Driven Evaluation and Refinement , which allocates sparsity incrementally. At each step, it evaluates pruning an additional fraction of weights from each eligible layer and commits the layer update that causes the least end-to-end degradation per removed parameter. This process produces non-uniform sparsity profiles and exactly nested masks across multiple budgets. We evaluate LADDER on five OPT, Llama, and Qwen models across 10-90% sparsity, using SparseGPT as its candidate-generation backend and comparing against uniform and adaptive allocation baselines. Results reveal a clear operating-regime trade-off: LADDER is comparable or slightly weaker at benign sparsity but becomes substantially more robust from 70% onward. This high-sparsity advantage largely persists after posterior distillation across four of five models and translates into improved retention of aggregate zero-shot accuracy.