Unsloth says it cannot reproduce GSQ-RCO's recovery numbers and wants the methods

Unsloth co-founder Daniel Han said on November 2nd, 2026 that Unsloth measured GSQ-RCO's IQ1_M quantization at 97.24% recovery, against ISTA-DASLab's claimed 98.7%, and that the GSQ-RCO team told Unsloth to reproduce the benchmarks itself rather than share its methods. Unsloth published run counts, context lengths, graders and confidence intervals for its rerun, which put recovery at 97.24% to 99.54% versus ISTA-DASLab's claimed 98.7% to 100%, and Han is asking ISTA-DASLab to publish its benchmark protocol so third parties can settle the numbers. The gap matters because developers pick quantized GGUF files for local DeepSeek and Qwen runs on published recovery scores.

Unsloth says it cannot reproduce GSQ-RCO's recovery numbers and wants the methods Daniel Han says Unsloth measured GSQ-RCO IQ1 M at 97.24% recovery against a claimed 98.7%, and wants ISTA-DASLab's benchmark protocol published. By Ryan Merket https://runtimewire.com/author/ryan-merket · Published Primary source: x.com https://x.com/danielhanchen/status/2117298764994204009 Why it matters Quantized GGUF files are chosen on published recovery scores, so a public gap between ISTA-DASLab's claimed 98.7 to 100% recovery and Unsloth's measured 97.24 to 99.54% decides which files developers download for local DeepSeek and Qwen runs. The dispute also sets a practical standard: Unsloth has posted run counts, context lengths, graders and confidence intervals for its rerun, and the same disclosure from ISTA-DASLab would let any third party settle the numbers instead of picking a side. Daniel Han @danielhanchen https://x.com/danielhanchen , Unsloth's co-founder, said on November 2nd, 2026 that Unsloth asked the GSQ-RCO team for more detail on how its benchmarks were made, and got told to reproduce the numbers itself. In the post on X https://x.com/danielhanchen/status/2117298764994204009 , Han reproduced comments he attributed to a now-deleted Reddit account: "Please just try to reproduce our numbers yourself. Even if your hardware doesn't allow you to run DeepSeek, you can still check our benchmarks on QFN and Qwen3.8-27B https://runtimewire.com/models/qwen/qwen3.8-27b .… When you purchase through links in our articles, we may earn a small commission https://runtimewire.com/affiliate-monetization-standards . This doesn’t affect our editorial independence.