LLM Quantization Methods: A Comprehensive Comparative Analysis

Large language model quantization has evolved from a niche optimization technique into a critical deployment infrastructure. This report analyzes 15+ quantization methods across three dimensions: (1) algorithmic methodology, (2) performance characteristics, and (3) ecosystem maturity. The landscape divides into four primary paradigms: CPU-optimized GGUF-based approaches (llama.cpp k-quants, I-quants, curated by bartowski), GPU-native weight-only kernels (AWQ, GPTQ with Marlin/EXL2 backends), NVIDIA’s floating-point hierarchy (FP8, NVFP4 on Blackwell hardware, rooted in the APEX mixed-precision legacy), and novel calibration-free methods (HQQ, AutoRound).