Shapelearn Qwen 3.8 27B (13.1 GB VRAM)

ByteShape released full ShapeLearn GGUF quantizations for Qwen 3.8 27B, with all five models sitting on the measured quality-speed frontier across six GPU comparisons, the company reported. The release follows the earlier ShapeLearn-Lite GGUFs published on August 18, 2026, four days after Qwen 3.8 27B's August 14, 2026 launch, and ByteShape said speculative decoding with MTP or DFlash2 raised throughput on every ShapeLearn model and GPU tested, with DFlash2 usually faster but requiring more memory and lacking image-input support in llama.cpp. DFlash2 requires llama.cpp b10658 or newer and uses a separate 1.1 GB draft model, while the MTP draft head is bundled in every GGUF.

ShapeLearn-Lite Held Up. ShapeLearn Did Better: Qwen 3.8 27B We were a little impatient. Qwen 3.8 27B was released on August 14, 2026. Four days later, on August 18, we published our first set of GGUFs. We called them ShapeLearn-Lite for a reason: they were produced using a much smaller optimization budget, fewer checks, and much less waiting. Now the full ShapeLearn models are done, and we have benchmarked them alongside the original Lite set and competing quants. The good news: ShapeLearn-Lite held up pretty well. We will come back to that later in “ShapeLearn-Lite, in retrospect” shapelearn-lite-in-retrospect . The better news: the full ShapeLearn models are even better. The MTP draft head is bundled in every GGUF. DFlash2 uses a separate 1.1 GB draft model. Both commands use GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf llama-server \ -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4 XS-3.84bpw \ --mmproj-auto \ --spec-type draft-mtp --spec-draft-n-max 3 llama-server \ -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4 XS-3.84bpw \ -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4 K M \ --spec-type draft-dflash --spec-draft-n-max 7 \ --no-mmproj DFlash2 needs llama.cpp b10658 or newer. Ready-to-run commands for every model, with the recommended sampling settings, are in the run tool https://byteshape.com/run-hf-model/?tag=byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4 XS-3.84bpw&platform=llamacpp and on the model card https://huggingface.co/byteshape/Qwen3.8-27B-GGUF . TL;DR - Full ShapeLearn moves the measured quality-speed frontier beyond Lite. All five models in the new release sit on the frontier in each of our six GPU comparisons. GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf - ShapeLearn-Lite also performed better than its KLD ranking suggested: three of its six models sit on the frontier in the Lite-versus-Unsloth Dynamic v3 comparison. - Speculative Decoding with MTP or DFlash2 increases throughput across every ShapeLearn model and GPU tested. DFlash2 is usually faster but requires more memory and does not support image inputs with llama.cpp. Choose DFlash2 for maximum text-only throughput when memory allows, and MTP when VRAM or multimodal support matters more. Full ShapeLearn moves the frontier We are releasing the full ShapeLearn run for Qwen 3.8 27B. Within this release, larger models yield higher aggregate scores, while smaller models deliver higher throughput. That ordering holds across all six GPUs tested. Because this is a dense model and memory transfers are the bottleneck, lower BPW translates more directly into higher TPS than it does for MoEs. The per-GPU comparisons also include AtomicChat https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF , Bartowski https://huggingface.co/bartowski/Qwen3.8-27B-GGUF , ISTA-DASLab https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF , and Unsloth Dynamic v3 https://huggingface.co/unsloth/Qwen3.8-27B-GGUF . Bartowski’s latest models were released after our testing and are not included. Full ShapeLearn is labelled ByteShape in the figures. All five ShapeLearn models remain on the measured frontier, with GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf By “frontier,” we mean that no other plotted model is both faster and more accurate. 96 GB: RTX Pro 6000 RTX PRO 6000, the GPU with the most memory, lets us show the full range of models tested. Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 116.11 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 108.01 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 105.44 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 101.11 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 90.42 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 114.81 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 106.95 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 100.51 | 3.14 | | D | UD-IQ3 S | 0.9555 | 95.53 | 3.47 | | E | UD-Q3 K XL | 0.9760 | 90.88 | 3.80 | | F | UD-IQ4 XS | 0.9920 | 86.73 | 4.13 | | G | UD-Q4 K S | 0.9877 | 82.21 | 4.46 | | H | UD-Q4 K M | 0.9703 | 78.58 | 4.79 | | I | UD-Q4 K XL | 0.9871 | 74.75 | 5.12 | | J | UD-Q5 K S | 0.9878 | 71.00 | 5.44 | | K | UD-Q5 K M | 0.9897 | 67.71 | 5.77 | | L | UD-Q5 K XL | 0.9905 | 65.52 | 6.10 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 112.77 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 108.22 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 103.39 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 94.77 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 112.21 | 2.72 | | b | IQ2 S | 0.9296 | 106.33 | 2.99 | | c | Q2 K | 0.9616 | 96.08 | 3.45 | | d | IQ3 XXS | 0.9594 | 92.43 | 3.68 | | e | IQ3 XS | 0.9582 | 88.36 | 3.89 | | f | IQ3 M | 0.9667 | 86.17 | 4.06 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 116.28 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 108.67 | 2.85 | | c | AD-IQ2 S | 0.9061 | 100.28 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 95.29 | 3.50 | | e | AD-IQ3 S | 0.9730 | 88.52 | 4.04 | GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf 32 GB: RTX 5090 The RTX 5090 tells a similar story, leading to the same recommendations. Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 119.13 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 110.78 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 108.08 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 103.57 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 93.66 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 115.34 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 108.12 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 102.87 | 3.14 | | D | UD-IQ3 S | 0.9555 | 97.87 | 3.47 | | E | UD-Q3 K XL | 0.9760 | 93.46 | 3.80 | | F | UD-IQ4 XS | 0.9920 | 89.69 | 4.13 | | G | UD-Q4 K S | 0.9877 | 85.28 | 4.46 | | H | UD-Q4 K M | 0.9703 | 81.63 | 4.79 | | I | UD-Q4 K XL | 0.9871 | 77.51 | 5.12 | | J | UD-Q5 K S | 0.9878 | 73.60 | 5.44 | | K | UD-Q5 K M | 0.9897 | 69.87 | 5.77 | | L | UD-Q5 K XL | 0.9905 | 67.59 | 6.10 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 114.11 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 110.19 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 105.67 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 95.49 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 115.77 | 2.72 | | b | IQ2 S | 0.9296 | 109.39 | 2.99 | | c | Q2 K | 0.9616 | 99.29 | 3.45 | | d | IQ3 XXS | 0.9594 | 95.83 | 3.68 | | e | IQ3 XS | 0.9582 | 91.14 | 3.89 | | f | IQ3 M | 0.9667 | 89.08 | 4.06 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 119.53 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 112.39 | 2.85 | | c | AD-IQ2 S | 0.9061 | 103.46 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 98.22 | 3.50 | | e | AD-IQ3 S | 0.9730 | 91.72 | 4.04 | Once again GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf 24 GB: RTX 4090 and RTX 3090 Both 24 GB cards fit all five ShapeLearn models. We plot them separately because their throughput differs, but the ordering is the same on both. RTX 4090 The RTX 4090 keeps the same pattern: GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 78.67 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 72.89 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 71.12 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 67.77 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 59.16 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 78.23 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 72.79 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 67.24 | 3.14 | | D | UD-IQ3 S | 0.9555 | 63.02 | 3.47 | | E | UD-Q3 K XL | 0.9760 | 59.13 | 3.80 | | F | UD-IQ4 XS | 0.9920 | 55.62 | 4.13 | | G | UD-Q4 K S | 0.9877 | 52.44 | 4.46 | | H | UD-Q4 K M | 0.9703 | 49.77 | 4.79 | | I | UD-Q4 K XL | 0.9871 | 47.08 | 5.12 | | J | UD-Q5 K S | 0.9878 | 44.80 | 5.44 | | K | UD-Q5 K M | 0.9897 | 42.45 | 5.77 | | L | UD-Q5 K XL | 0.9905 | 40.84 | 6.10 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 77.80 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 73.34 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 69.26 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 62.41 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 75.28 | 2.72 | | b | IQ2 S | 0.9296 | 71.27 | 2.99 | | c | Q2 K | 0.9616 | 63.74 | 3.45 | | d | IQ3 XXS | 0.9594 | 61.03 | 3.68 | | e | IQ3 XS | 0.9582 | 58.20 | 3.89 | | f | IQ3 M | 0.9667 | 56.16 | 4.06 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 79.51 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 74.19 | 2.85 | | c | AD-IQ2 S | 0.9061 | 67.40 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 63.60 | 3.50 | | e | AD-IQ3 S | 0.9730 | 57.38 | 4.04 | RTX 3090 Older, but still fast in these measurements. Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 53.03 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 51.20 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 50.75 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 49.49 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 45.79 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 52.82 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 50.37 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 48.16 | 3.14 | | D | UD-IQ3 S | 0.9555 | 46.39 | 3.47 | | E | UD-Q3 K XL | 0.9760 | 46.28 | 3.80 | | F | UD-IQ4 XS | 0.9920 | 46.12 | 4.13 | | G | UD-Q4 K S | 0.9877 | 44.49 | 4.46 | | H | UD-Q4 K M | 0.9703 | 43.30 | 4.79 | | I | UD-Q4 K XL | 0.9871 | 41.37 | 5.12 | | J | UD-Q5 K S | 0.9878 | 39.37 | 5.44 | | K | UD-Q5 K M | 0.9897 | 37.38 | 5.77 | | L | UD-Q5 K XL | 0.9905 | 36.16 | 6.10 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 51.56 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 50.07 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 48.66 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 47.19 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 53.63 | 2.72 | | b | IQ2 S | 0.9296 | 51.21 | 2.99 | | c | Q2 K | 0.9616 | 45.34 | 3.45 | | d | IQ3 XXS | 0.9594 | 47.46 | 3.68 | | e | IQ3 XS | 0.9582 | 44.68 | 3.89 | | f | IQ3 M | 0.9667 | 43.42 | 4.06 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 53.95 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 51.91 | 2.85 | | c | AD-IQ2 S | 0.9061 | 48.24 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 47.10 | 3.50 | | e | AD-IQ3 S | 0.9730 | 47.22 | 4.04 | GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf 16 GB: RTX 4080 and RTX 5060 Ti With a tighter VRAM budget, the pragmatic choice is to leave room for the context you need, not just the model weights. These plots contain fewer competing configurations, but all five ShapeLearn models are represented. RTX 4080 On the RTX 4080, GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 62.01 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 56.84 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 55.14 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 52.43 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 45.74 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 61.70 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 56.70 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 52.32 | 3.14 | | D | UD-IQ3 S | 0.9555 | 49.06 | 3.47 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 60.47 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 57.37 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 54.19 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 48.42 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 58.69 | 2.72 | | b | IQ2 S | 0.9296 | 55.39 | 2.99 | | c | Q2 K | 0.9616 | 48.91 | 3.45 | | d | IQ3 XXS | 0.9594 | 46.99 | 3.68 | | e | IQ3 XS | 0.9582 | 44.71 | 3.89 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 62.55 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 57.82 | 2.85 | | c | AD-IQ2 S | 0.9061 | 52.43 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 49.15 | 3.50 | RTX 5060 Ti On the RTX 5060 Ti, the corresponding figures are 33.1 tok/s and 29.1 tok/s. Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ByteShape | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 38.05 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 35.41 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 34.50 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 33.05 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 29.15 | 3.84 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 38.02 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 35.42 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 32.80 | 3.14 | | D | UD-IQ3 S | 0.9555 | 30.97 | 3.47 | | ISTA-DASLab | | | | | | a | GSQ-RCO-IQ2 XS | 0.8647 | 37.62 | 2.50 | | b | GSQ-RCO-IQ2 S | 0.9364 | 35.80 | 2.75 | | c | GSQ-RCO-IQ3 XXS | 0.9438 | 33.95 | 3.00 | | d | GSQ-RCO-IQ3 S | 0.9943 | 30.72 | 3.50 | | Bartowski | | | | | | a | IQ2 XXS | 0.7986 | 36.33 | 2.72 | | b | IQ2 S | 0.9296 | 34.76 | 2.99 | | c | Q2 K | 0.9616 | 30.79 | 3.45 | | d | IQ3 XXS | 0.9594 | 29.69 | 3.68 | | e | IQ3 XS | 0.9582 | 28.24 | 3.89 | | AtomicChat | | | | | | a | AD-IQ2 XXS | 0.8385 | 38.42 | 2.58 | | b | AD-IQ2 XS | 0.9296 | 36.05 | 2.85 | | c | AD-IQ2 S | 0.9061 | 32.61 | 3.22 | | d | AD-IQ3 XXS | 0.9644 | 31.01 | 3.50 | GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf ShapeLearn-Lite, in retrospect ShapeLearn-Lite uses a smaller optimization budget than full ShapeLearn. It let us get Qwen 3.8 27B onto 12 GB to 24 GB GPUs within a few days. We released after targeted sanity checks and started the full evaluation afterwards. The full ShapeLearn models were ready before the benchmarking was finished. Evaluating both sets, along with the competing models, is what took most of the time. Then Unsloth released its Dynamic v3 models. At similar sizes, several had lower KLD than Lite in our measurements. On KLD alone, Lite looked less competitive. KLD looked decisive KLD measures divergence between a quantized model’s predicted token distributions and the BF16 reference under a particular evaluation setup. It is useful for diagnosing substantial changes, but lower divergence does not automatically mean better task performance. We measure KLD on a dataset of about 5 million tokens of prompt and response pairs, drawn from several benchmarks, including long-context and agentic tasks. We also changed how KLD is computed, so that it is closer to what we expect KLD to measure: - KLD is measured on response tokens only, not on prompt tokens. We do not want to measure how well a model can generate prompts. - KLD only considers the tokens that have a chance of being sampled during generation, the top-20, top-40, or top-60 tokens at each position. The tail tokens never get sampled, so they do not contribute. - Requests have clear boundaries. Each prompt and response pair is scored as its own request, not as part of one long concatenated stream. Show Legend | | Model | KLD | Size GB | BPW | |---|---|---|---|---| | ShapeLearn-Lite | | | | | | Lite-1 | IQ3 S-3.44bpw | 0.035875 | 10.79 | 3.44 | | Lite-2 | IQ4 XS-3.67bpw | 0.028296 | 11.51 | 3.68 | | Lite-3 | IQ4 XS-4.00bpw | 0.018249 | 12.52 | 4.00 | | Lite-4 | IQ4 XS-4.40bpw | 0.009901 | 13.78 | 4.40 | | Lite-5 | Q5 K S-4.72bpw | 0.007578 | 14.78 | 4.72 | | Lite-6 | Q5 K M-5.60bpw | 0.003297 | 17.53 | 5.60 | | Unsloth | | | | | | i | UD-IQ1 S | 0.389550 | 5.76 | 1.84 | | ii | UD-IQ1 M | 0.261876 | 6.26 | 2.00 | | iii | UD-IQ2 XXS | 0.181493 | 6.76 | 2.16 | | iv | UD-IQ2 S | 0.108374 | 7.79 | 2.49 | | v | UD-Q2 K XL | 0.065200 | 8.81 | 2.81 | | vi | UD-IQ3 XXS | 0.040407 | 9.84 | 3.14 | | vii | UD-IQ3 S | 0.028759 | 10.87 | 3.47 | | viii | UD-Q3 K XL | 0.019844 | 11.90 | 3.80 | | ix | UD-IQ4 XS | 0.011992 | 12.93 | 4.13 | | x | UD-Q4 K S | 0.008827 | 13.96 | 4.46 | | xi | Q4 0 | 0.019264 | 14.69 | 4.69 | | xii | UD-Q4 K M | 0.007054 | 14.99 | 4.79 | | xiii | UD-Q4 K XL | 0.005210 | 16.01 | 5.11 | | xiv | Q4 1 | 0.009603 | 16.06 | 5.13 | | xv | UD-Q5 K S | 0.003619 | 17.04 | 5.44 | | xvi | UD-Q5 K M | 0.002839 | 18.07 | 5.77 | | xvii | UD-Q5 K XL | 0.002432 | 19.10 | 6.10 | | xviii | UD-Q6 K | 0.001771 | 20.13 | 6.43 | | xix | UD-Q6 K M | 0.001426 | 21.16 | 6.76 | | xx | UD-Q6 K L | 0.001150 | 22.19 | 7.09 | | xxi | UD-Q6 K XL | 0.000985 | 23.22 | 7.42 | | xxii | UD-Q8 K L | 0.000726 | 25.78 | 8.23 | | xxiii | Q8 0 | 0.000648 | 26.62 | 8.50 | | xxiv | UD-Q8 K XL | 0.000503 | 28.76 | 9.19 | For example, Unsloth’s UD-IQ3 S vii has about 20% lower KLD than the similarly sized smallest Lite model Lite-1 : 0.028759 versus 0.035875. Yet its aggregate benchmark score is lower: 95.55% versus 97.33% of BF16. If lower KLD were sufficient to rank these models by task performance, the benchmark ordering should have followed it. It did not. The point is not that KLD is useless. It is that a fidelity ranking is not a task-performance ranking. This is the distinction explored in our KLD evaluation blog https://byteshape.com/blogs/Evaluating-Quantized-Models/part-2/ . Our related paper on KLD and quantization fidelity metrics https://arxiv.org/abs/2606.19558 was also recently accepted to the EMNLP https://2026.emnlp.org/ Industry Track. Lite held up Naturally, we made more plots. Here, we show the RTX Pro 6000 because it can accommodate the full comparison. Each model’s benchmark score is reused across the GPU plots; the measured throughput and the set of displayed models change. Show Legend | | Model | Acc | TPS | BPW | |---|---|---|---|---| | ShapeLearn this release | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 116.11 | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 108.01 | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 105.44 | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 101.11 | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 90.42 | 3.84 | | ShapeLearn-Lite | | | | | | Lite-1 | IQ3 S-3.44bpw | 0.9733 | 98.74 | 3.45 | | Lite-2 | IQ4 XS-3.67bpw | 0.9802 | 94.48 | 3.68 | | Lite-3 | IQ4 XS-4.00bpw | 0.9880 | 89.85 | 4.00 | | Lite-4 | IQ4 XS-4.40bpw | 0.9856 | 85.03 | 4.40 | | Lite-5 | Q5 K S-4.72bpw | 0.9909 | 79.75 | 4.72 | | Lite-6 | Q5 K M-5.60bpw | 0.9919 | 70.28 | 5.60 | | Unsloth | | | | | | A | UD-IQ2 S | 0.8633 | 114.81 | 2.49 | | B | UD-Q2 K XL | 0.9572 | 106.95 | 2.82 | | C | UD-IQ3 XXS | 0.9359 | 100.51 | 3.14 | | D | UD-IQ3 S | 0.9555 | 95.53 | 3.47 | | E | UD-Q3 K XL | 0.9760 | 90.88 | 3.80 | | F | UD-IQ4 XS | 0.9920 | 86.73 | 4.13 | | G | UD-Q4 K S | 0.9877 | 82.21 | 4.46 | | H | UD-Q4 K M | 0.9703 | 78.58 | 4.79 | | I | UD-Q4 K XL | 0.9871 | 74.75 | 5.12 | | J | UD-Q5 K S | 0.9878 | 71.00 | 5.44 | | K | UD-Q5 K M | 0.9897 | 67.71 | 5.77 | | L | UD-Q5 K XL | 0.9905 | 65.52 | 6.10 | Leaving the full ShapeLearn models aside for a moment, three of the six ShapeLearn-Lite models sit on the Lite-versus-Unsloth frontier: the three smallest Lite models, the lighter orange bubbles labelled 1-3. Of the twelve Unsloth v3 models shown, three also sit on that frontier: UD-IQ2 S A , UD-Q2 K XL B , and UD-IQ4 XS F . UD-IQ4 XS F is a strong higher-quality point, while Lite earns its places in the middle of the range. Add the five full ShapeLearn models back in the darker orange bubbles , and they take over the entire frontier. Lite was never meant to be the final result. It still held its own where it mattered. Speculative Decoding We also evaluated MTP and DFlash2 with the new models, using 3 draft tokens for MTP and 7 draft tokens for DFlash2. Both methods increased throughput for all five ShapeLearn models on all six GPUs tested. DFlash2 was faster than MTP in almost all cases. Across the full lineup, DFlash2 reached 1.34-2.10x the baseline next-token prediction NTP throughput, while MTP reached 1.28-1.66x. We measured with the sampling parameters Qwen recommends for thinking mode, over a diverse set of agentic coding, mathematics, and general-knowledge requests. The speedups would likely be larger under greedy decoding, but temperature-based sampling better reflects real usage. The figure below shows NTP, MTP, and DFlash2 throughput for each GPU. The quality axis is the target-model benchmark score reported above. These plots do not independently establish quality equivalence between decoding methods. Show Legend | | Model | Acc | NTP TPS | MTP TPS | DFlash2 TPS | BPW | |---|---|---|---|---|---|---| | RTX Pro 6000 96 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 116.11 | 165.52 1.43x | 172.01 1.48x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 108.01 | 153.48 1.42x | 165.94 1.54x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 105.44 | 152.19 1.44x | 166.03 1.57x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 101.11 | 146.85 1.45x | 164.24 1.62x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 90.42 | 145.69 1.61x | 150.83 1.67x | 3.84 | | RTX 5090 32 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 119.13 | 164.44 1.38x | 175.53 1.47x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 110.78 | 156.63 1.41x | 175.97 1.59x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 108.08 | 155.19 1.44x | 171.97 1.59x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 103.57 | 148.34 1.43x | 169.69 1.64x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 93.66 | 147.00 1.57x | 166.57 1.78x | 3.84 | | RTX 4090 24 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 78.67 | 111.73 1.42x | 135.39 1.72x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 72.89 | 108.40 1.49x | 132.20 1.81x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 71.12 | 107.21 1.51x | 133.38 1.88x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 67.77 | 101.47 1.50x | 131.12 1.93x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 59.16 | 98.25 1.66x | 124.15 2.10x | 3.84 | | RTX 3090 24 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 53.03 | 68.19 1.29x | 70.94 1.34x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 51.20 | 65.76 1.28x | 68.90 1.35x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 50.75 | 65.82 1.30x | 68.40 1.35x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 49.49 | 64.50 1.30x | 66.34 1.34x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 45.79 | 66.55 1.45x | 63.93 1.40x | 3.84 | | RTX 4080 16 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 62.01 | 87.06 1.40x | 101.63 1.64x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 56.84 | 81.25 1.43x | 96.90 1.70x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 55.14 | 79.74 1.45x | 96.95 1.76x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 52.43 | 76.88 1.47x | 94.09 1.79x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 45.74 | 74.01 1.62x | 87.49 1.91x | 3.84 | | RTX 5060 Ti 16 GB | | | | | | | | GPU-1 | IQ2 XXS-2.56bpw | 0.9304 | 38.05 | 50.54 1.33x | 56.71 1.49x | 2.56 | | GPU-2 | IQ3 XXS-2.88bpw | 0.9656 | 35.41 | 47.94 1.35x | 52.57 1.48x | 2.88 | | GPU-3 | IQ3 XS-3.01bpw | 0.9726 | 34.50 | 47.93 1.39x | 52.28 1.52x | 3.01 | | GPU-4 | IQ3 S-3.23bpw | 0.9872 | 33.05 | 46.25 1.40x | 50.34 1.52x | 3.23 | | GPU-5 | IQ4 XS-3.84bpw | 0.9963 | 29.15 | 45.82 1.57x | 47.01 1.61x | 3.84 | There is also a memory tradeoff between the two approaches. The embedded quantized MTP weights add only about 250 MB to the model, and if MTP is not used, these weights are not loaded into GPU memory. In comparison, the 4-bit DFlash2 draft model is about 1.1 GB, so enabling DFlash2 requires roughly 1.1 GB of additional GPU memory. Packaging MTP as a separate GGUF file would largely eliminate this advantage. The standalone model would need its own MTP embedding and output layers, which are by far its largest tensors, bringing its memory footprint to roughly 1 GB as well. In addition, DFlash2 in llama.cpp currently does not support image inputs, which is an important consideration for multimodal use cases. Benchmarking Methodology We evaluate all reported models across a set of instruct and thinking benchmarks. Instruct benchmarks: - GSM8K for math - IFEval for instruction following - MMLU for general knowledge - LiveCodeBench V6 for coding - Multi-IF for multi-turn and multilingual instruction following - ACEBench for tool use and agentic tasks Thinking benchmarks: - ACEBench for tool use and agentic tasks - Multiple HumanEval for coding - BFCL V4 for tool calling and agentic tasks For the thinking benchmarks, we used Qwen 3.8’s medium thinking setting. For each benchmark, the score of a quantized model is normalized by the score of the corresponding BF16 model. The overall reported score is the average of these normalized benchmark scores. Our LiveCodeBench V6 evaluation includes problems from January 1, 2024 onward, excluding the 2023 problems. We found the 2023 problems to be relatively easy for current models, with most models achieving very high scores on them. As a result, they provide limited discrimination between models while adding substantial evaluation time. For BFCL V4 , we evaluate the following eight subsets: - live simple - live parallel - live parallel multiple - live relevance - multi turn base - multi turn miss func - multi turn miss param - multi turn long context All evaluations were run with llama.cpp b10430. For both instruct and thinking experiments, we use the sampling parameters recommended by Qwen for the corresponding mode. Conclusion ShapeLearn-Lite did what it was designed to do. It got useful Qwen 3.8 27B quants onto 12 to 24 GB GPUs quickly, and it held up better than its KLD ranking suggested. Full ShapeLearn goes further. It improves the measured quality-speed trade-offs over Lite and contributes five frontier models across all six tested GPUs. GPU-5 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4 XS-3.84bpw.gguf GPU-4 https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3 S-3.23bpw.gguf KLD remains useful, but it is not a task-performance leaderboard. Fidelity metrics tell us how much the model’s distributions changed under a particular measurement. Benchmarks tell us whether those changes matter on the tasks we tested. We were impatient. This time, it worked out pretty well.