{"slug": "archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less", "title": "Archsloth fixes two AutoRound flags, says Qwen3-4B drifts less", "summary": "Archsloth published a September 14th correction to its AutoRound quantization workflow, reporting that two changed settings produced a Qwen3-4B GGUF file with 20.9% to 54.4% lower KL divergence than Unsloth's Qwen3-4B Q4_K_M build at the identical 2,497,280,800-byte size. The correction, written by SeaWolf-AI in the FINAL-Bench community article on Hugging Face, fixed a W4A16 scheme passed to AutoRound while exporting as gguf:q4_k_m and a second flag, with reported reductions of 52.9% for source code, 33.7% for Japanese, 33.2% for English and 31.1% for Chinese across 10 axes covering seven writing systems. Archsloth said inference speed was essentially tied and noted KL divergence does not establish matching gains in downstream task accuracy, safety or application quality.", "body_md": "# Archsloth fixes two AutoRound flags, says Qwen3-4B drifts less\n\n**The project reports 20.9% to 54.4% lower KL divergence at the same 2.50 GB size, with calibration text doing more work than bit allocation.**\n\n        By [RuntimeWire Staff](/author/runtimewire-staff)\n        · Published \n\nPrimary source: [Hugging Face Newsroom](https://huggingface.co/blog/FINAL-Bench/qwen-models)\n\n## Why it matters\n\nLocal-model distributors can ship files with the same size and format that preserve materially different behavior. Quantization recipes and calibration data deserve the same scrutiny as model weights and benchmarks.\n\nThe contributors behind [Archsloth](https://huggingface.co/Archsloth?ref=runtimewire) published a correction on September 14th to a quantization workflow they had used for months, saying two changed AutoRound settings produced a version of Alibaba's Qwen3-4B that stayed closer to the original model without adding a single byte.\n\nThe work came from [SeaWolf-AI](https://huggingface.co/SeaWolf-AI?ref=runtimewire), writing through the [FINAL-Bench community article](https://huggingface.co/blog/FINAL-Bench/qwen-models?ref=runtimewire) on Hugging Face. Archsloth publishes quantized model files under the Archsloth name.\n\nThat approach matters here because Archsloth is making a narrow, testable claim. Its [Qwen3-4B GGUF repository](https://huggingface.co/Archsloth/Qwen3-4B-GGUF?ref=runtimewire) contains the calibration corpus, evaluation texts and per-run logs used for the comparison. The project also publishes the commands needed to download the competing file, convert the original BF16 weights and rerun its KL-divergence measurements.\n\n### The same format concealed a different rounding process\n\nQuantization compresses a model's weights into lower-precision values, reducing the memory and storage required for local inference. Developers commonly choose files by model name, quantization label and size. Archsloth's test shows why those three labels can conceal a consequential part of the build process.\n\nArchsloth compared its Q4_K_M file with [Unsloth's Qwen3-4B Q4_K_M repository](https://huggingface.co/unsloth/Qwen3-4B-GGUF?ref=runtimewire). Both files were exactly 2,497,280,800 bytes. Both used the same architecture and the same tensor map: 216 Q4_K tensors, 37 Q6_K tensors and 145 F32 tensors. Both ran through stock llama.cpp.\n\nThe integer selected for each rounded weight was the reported difference.\n\nArchsloth measured that difference using KL divergence from Alibaba's original BF16 model. Lower divergence means the compressed model's output distribution remained closer to the reference model on the evaluation text. Across 10 axes covering seven writing systems, Archsloth reported lower divergence ranging from 20.9% for Hindi to 54.4% for Korean. The reported reductions included 52.9% for source code, 33.7% for Japanese, 33.2% for English and 31.1% for Chinese.\n\nThree axes - Korean, English and source code - appeared in Archsloth's calibration set. The remaining seven served as checks against buying an improvement in the calibrated languages at the expense of others. The published results did not show that pattern on the 4B test.\n\nThese remain Archsloth's measurements. KL divergence captures how closely a quantized model preserves the reference model's output distribution. It does not establish a matching gain in downstream task accuracy, safety or application quality. Archsloth reported essentially tied inference speed, which fits the identical file size and tensor layout.\n\n### Two settings survived because nothing crashed\n\nThe first error sat in the quantization scheme. Archsloth had passed `W4A16` to AutoRound while exporting the result as `gguf:q4_k_m`. W4A16 and GGUF Q4_K_M use different grouping, scaling and zero-point rules. The search therefore optimized rounding decisions for one quantizer before the export process rebuilt them for another.\n\nThe correction changed the scheme to `GGUF:Q4_K_M`, matching the format used at export. [Intel's AutoRound documentation](https://github.com/intel/auto-round?ref=runtimewire) lists W4A16 and GGUF:Q4_K_M as separate supported schemes.\n\nThe second change enabled `--enable_alg_ext`, an experimental AutoRound option that is disabled by default. Archsloth linked the setting to [SignRoundV2](https://arxiv.org/abs/2512.04746?ref=runtimewire), a sign-gradient quantization method published by Intel researchers. The project said the option increased tuning time for the 4B model from about six minutes to 11 minutes on one H100.\n\nThe failure mode was mundane enough to persist. AutoRound produced a valid file, its optimization loss declined and llama.cpp could run the result. A working artifact gave the maintainers little reason to suspect that much of the rounding search was being discarded during export.\n\nFor developers building deployment tooling, that is the useful part of Archsloth's correction. A successful conversion only proves that a file can be loaded. It says little about whether the optimization procedure actually targeted the format that shipped.\n\n### Calibration text became a product decision\n\nArchsloth's more consequential finding concerned the text used during quantization. AutoRound chooses whether individual weights round up or down by minimizing reconstruction error on calibration activations. The calibration corpus therefore participates directly in those choices.\n\nIn a controlled [Qwen3.5-9B](/models/qwen/qwen3.5-9b:batch) comparison, Archsloth reported a Korean mean KL divergence of 0.019255 with interleaved Korean and English calibration samples, compared with 0.027336 using English alone. That amounted to a reported 29.6% reduction from changing the text while holding the model, code, settings, bit widths and byte count constant. A 27B experiment produced a reported 31.9% difference.\n\nSample order also mattered. Archsloth said alternating Korean and English samples improved the English result by 11.4% compared with grouping the languages, despite keeping the overall character mix constant. Adding source code improved the code axis by 52.9%, while Korean and English results deteriorated by 3.4% and 5.3%, respectively. Archsloth accepted and documented that trade.\n\nThe project also published failed experiments. Four times as many calibration samples increased compute cost without moving the result. Quantization-aware healing worsened the measured divergence by 57% after the corrected rounding search was enabled. Per-layer bit allocation ranged from no measurable gain to a 10% deterioration.\n\nThose results deepen a pattern RuntimeWire covered in [Bartowski's work on per-tensor GGUF maps](/article/bartowski-per-tensor-layout-maps-gguf-quantization) on September 11th. Local model packaging is becoming an engineering discipline of its own. Tensor-level bit choices, calibration samples, optimization settings and evaluation methods can alter the artifact even when the model card presents the same familiar quantization label.\n\nAlibaba released [Qwen3](https://qwenlm.github.io/blog/qwen3/?ref=runtimewire) on April 29th, 2025, with dense and mixture-of-experts models intended for open deployment. The work happening downstream now determines which version many users actually run on laptops, workstations and local servers. Archsloth's contribution is a reminder that model distribution does not end when the original weights reach Hugging Face. The last few lines of the build command can still decide how much of the model survives the trip.", "url": "https://wpnews.pro/news/archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less", "canonical_source": "https://runtimewire.com/article/archsloth-qwen3-4b-autoround-quantization-flags", "published_at": "2026-09-14 11:59:39+00:00", "updated_at": "2026-09-14 12:11:34.203623+00:00", "lang": "en", "topics": ["ai-research", "large-language-models", "ai-tools", "mlops"], "entities": ["Archsloth", "AutoRound", "Qwen3-4B", "SeaWolf-AI", "Hugging Face", "Unsloth", "Alibaba", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less", "markdown": "https://wpnews.pro/news/archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less.md", "text": "https://wpnews.pro/news/archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less.txt", "jsonld": "https://wpnews.pro/news/archsloth-fixes-two-autoround-flags-says-qwen3-4b-drifts-less.jsonld"}}