# Looking for feedback: Do common quantization saliency metrics really measure weight importance? > Source: > Published: 2026-07-09 07:52:16+00:00 Four widely used methods for quantizing LLMs (AWQ, GPTQ, OWQ, SpQR) each score which weights matter most, then protect the important ones. That score has almost never been tested on its own. These methods get validated end to end: does the quantized model still work? If a method’s error-compensation step can patch up a bad pick, the model looks fine either way, so a wrong importance score never shows up in the final numbers in any of the papers. I tested the scores directly instead: perturb one weight channel at a time, hold everything else exact, and measure the real damage. The four criteria look like they agree with each other far more than chance would allow, but most of that agreement turns out to be mechanical, from sharing one input statistic, not from converging on the truth. Strip that shared statistic out and three of the four barely track real sensitivity anymore. A gradient-based score is the exception and holds up throughout. To be clear, I’m not saying that “AWQ and GPTQ don’t work.” Their compensation steps can cover for a bad pick, which is exactly why no one had measured the pick itself directly before. I tested this by knocking out one weight channel at a time in three small open LLMs (Qwen2.5-0.5B, Pythia-410M, SmolLM2-360M) and measuring the real damage with forward-KL divergence, at the actual deployment quantization setting (INT3, group-128). Every rule for how I’d judge the results was written down and locked before I ran anything. Paper + data: [Saliency Without Validation: A Measured Audit of Channel-Importance Criteria in Post-Training Quantization | Zenodo](https://doi.org/10.5281/zenodo.20791865) Code (MIT, full pipeline, 100 tests): A bit about me: I’m an independent researcher, working on this without a lab behind me, so it hasn’t had much outside scrutiny yet. That’s part of why I’m posting here. I’d genuinely like feedback: holes in the method, a control I missed, anything that doesn’t hold up. Separately, I want to get this onto arXiv (cs.LG) so it’s actually findable, but I need one endorsement and don’t have a network of eligible endorsers yet. If you’ve endorsed a cs.LG paper before, or know someone who has, and are willing to take a look, I’d really appreciate it. Endorsement just confirms the paper is on-topic for cs.LG; it’s not a review or approval of the results. Thanks for reading this far either way.