AutoCompress: Automated Hardware-Guided Model Compression

AutoCompress, a new automated hardware-guided model-compression method, targets LLM pruning against hardware- and workload-specific constraints on latency, throughput and memory rather than parameter counts. The approach addresses a gap in existing pruning workflows, which force practitioners to manually select model sizes to meet latency, throughput and GPU memory limits.

LLM pruning must preserve quality while meeting hardware- and workload-specific constraints on latency, throughput, and memory. Existing pruning workflows often target parameter counts, requiring practitioners to manually select model sizes that meet latency, throughput, and GPU memory limits. Howev