{"slug": "beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with", "title": "Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement", "summary": "Researchers proposed Cross-layer Activation-aware Sensitivity Allocation (CASA), a two-phase mixed-precision quantization method that replaces scalar sensitivity proxies with an activation-aware metric derived from the Kronecker-factored Hessian, according to a new arXiv paper (2609.25916v1). The authors prove that even the optimal scalar proxy incurs multiplicative distortion up to the square root of the product of the input- and output-side Hessian condition numbers, a bound ranging from 10^1 to 10^13 for typical LLM modules, which makes inter-module sensitivity ranking unreliable. Experiments across multiple LLMs and bit budgets show CASA achieves lower perplexity than the latest scalar-proxy baselines, especially below 3 bits per weight, with zero-shot accuracy gains tracking each model's average per-module condition number.", "body_md": "arXiv:2609.25916v1 Announce Type: new \nAbstract: Mixed-precision weight quantization is commonly formulated as a Multiple-Choice Knapsack Problem (MCKP), yet existing solvers rely on scalar sensitivity proxies that collapse each weight matrix's Hessian into a single number and treat every module independently. We prove that even the optimal scalar proxy incurs multiplicative distortion up to $\\sqrt{\\kappa(\\mathbf{A})\\kappa(\\mathbf{B})}$ relative to the full activation-aware quadratic, where $\\kappa(\\mathbf{A})$ and $\\kappa(\\mathbf{B})$ denote the condition numbers of the input- and output-side Hessian factors. This bound varies from $10^1$ to $10^{13}$ for typical LLM modules, making inter-module sensitivity ranking unreliable. To address these limitations, we propose Cross-layer Activation-aware Sensitivity Allocation (CASA), a two-phase method. In Stage 1, the scalar proxy is replaced by an activation-aware metric derived from the Kronecker-factored Hessian, reducing the MCKP to a form whose continuous relaxation admits a closed-form solution. In Stage 2, a cross-layer-aware local search evaluates bit-width updates using the end-to-end model loss. Experiments on multiple LLMs across different bit budgets show that CASA achieves lower perplexity than the latest scalar-proxy baselines, especially at ultra-low bit-widths ($<3$ bits per weight). Moreover, the performance gain in zero-shot accuracy tracks the per-model average condition-number over modules, confirming the distortion bound as a practical indicator of scalar-proxy failure.", "url": "https://wpnews.pro/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with", "canonical_source": "https://www.machinebrief.com/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-l-v7j5", "published_at": "2026-09-23 04:00:00+00:00", "updated_at": "2026-09-23 04:55:12.200492+00:00", "lang": "en", "topics": ["large-language-models", "machine-learning", "ai-research"], "entities": ["Cross-layer Activation-aware Sensitivity Allocation (CASA)", "arXiv", "Multiple-Choice Knapsack Problem (MCKP)", "Kronecker-factored Hessian"], "alternates": {"html": "https://wpnews.pro/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with", "markdown": "https://wpnews.pro/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with.md", "text": "https://wpnews.pro/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with.txt", "jsonld": "https://wpnews.pro/news/beyond-scalar-sensitivity-activation-aware-mixed-precision-llm-quantization-with.jsonld"}}