Researchers proposed OBC-Prune, an outcome-based calibration method for pruning large reasoning models that weights reasoning tokens by their causal contribution to correct answers rather than by unif…
Quantization and pruning techniques such as GPTQ, AWQ, GGUF, SparseGPT, and LoRA-based pruning enable running large language models on consumer hardware by reducing memory footprint, with 4-bit quanti…
Researchers introduced Wisp, Wisp+, and Whisper, a family of difference-informed pruning methods that preserve output differences in large language models, improving sparsity performance over existing…
In 2026, edge inference has advanced to the point where an 80B Qwen model runs in just 4.3GB of RAM on a MacBook Pro, and a 35B model runs on an iPhone 18 Pro. This is achieved through a combination o…
DigitalOcean published a tutorial on June 19 demonstrating how to compress large language models using SparseGPT and Wanda pruning methods for GPU cloud deployment, targeting reduced inference costs a…