cd /news/ai-tools/shapelearn-qwen-3-8-27b-13-1-gb-vram · home › topics › ai-tools › article
[ARTICLE · art-133247] src=byteshape.com ↗ pub= topic=ai-tools verified=true sentiment=↑ positive

Shapelearn Qwen 3.8 27B (13.1 GB VRAM)

ByteShape released full ShapeLearn GGUF quantizations for Qwen 3.8 27B, with all five models sitting on the measured quality-speed frontier across six GPU comparisons, the company reported. The release follows the earlier ShapeLearn-Lite GGUFs published on August 18, 2026, four days after Qwen 3.8 27B's August 14, 2026 launch, and ByteShape said speculative decoding with MTP or DFlash2 raised throughput on every ShapeLearn model and GPU tested, with DFlash2 usually faster but requiring more memory and lacking image-input support in llama.cpp. DFlash2 requires llama.cpp b10658 or newer and uses a separate 1.1 GB draft model, while the MTP draft head is bundled in every GGUF.

read26 min views1 publishedSep 18, 2026
Shapelearn Qwen 3.8 27B (13.1 GB VRAM)
Image: source

#

      ShapeLearn-Lite Held Up. ShapeLearn Did Better:

      Qwen 3.8 27B

We were a little impatient.

Qwen 3.8 27B was released on August 14, 2026. Four days later, on August 18, we published our first set of GGUFs. We called them ShapeLearn-Lite for a reason: they were produced using a much smaller optimization budget, fewer checks, and much less waiting.

Now the full ShapeLearn models are done, and we have benchmarked them alongside the original Lite set and competing quants.

        The good news: ShapeLearn-Lite held up pretty well. We will come back to that later in [“ShapeLearn-Lite, in retrospect”](#shapelearn-lite-in-retrospect).

The better news: the full ShapeLearn models are even better.

          The MTP draft head is bundled in every GGUF. DFlash2 uses a separate 1.1 GB draft model. Both commands use [GPU-5](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4_XS-3.84bpw.gguf)

llama-server \
  -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
  --mmproj-auto \
  --spec-type draft-mtp --spec-draft-n-max 3
llama-server \
  -hf byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw \
  -hfd incoai/Qwen3.8-27B-DFlash2-GGUF:Q4_K_M \
  --spec-type draft-dflash --spec-draft-n-max 7 \
  --no-mmproj

          DFlash2 needs llama.cpp b10658 or newer. Ready-to-run commands for every model, with the recommended sampling settings, are in the [run tool](https://byteshape.com/run-hf-model/?tag=byteshape/Qwen3.8-27B-GGUF:Qwen3.8-27B-IQ4_XS-3.84bpw&platform=llamacpp) and on the [model card](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF).

TL;DR #

  • Full ShapeLearn moves the measured quality-speed frontier beyond Lite. All five models in the new release sit on the frontier in each of our six GPU comparisons. GPU-5 GPU-4
  • ShapeLearn-Lite also performed better than its KLD ranking suggested: three of its six models sit on the frontier in the Lite-versus-Unsloth Dynamic v3 comparison.
  • Speculative Decoding with MTP or DFlash2 increases throughput across every ShapeLearn model and GPU tested. DFlash2 is usually faster but requires more memory and does not support image inputs with llama.cpp. Choose DFlash2 for maximum text-only throughput when memory allows, and MTP when VRAM or multimodal support matters more.

Full ShapeLearn moves the frontier #

We are releasing the full ShapeLearn run for Qwen 3.8 27B.

Within this release, larger models yield higher aggregate scores, while smaller models deliver higher throughput. That ordering holds across all six GPUs tested. Because this is a dense model and memory transfers are the bottleneck, lower BPW translates more directly into higher TPS than it does for MoEs.

        The per-GPU comparisons also include [AtomicChat](https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF), [Bartowski](https://huggingface.co/bartowski/Qwen3.8-27B-GGUF), [ISTA-DASLab](https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF), and [Unsloth Dynamic v3](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF). Bartowski’s latest models were released after our testing and are not included. Full ShapeLearn is labelled ByteShape in the figures.
      

        All five ShapeLearn models remain on the measured frontier, with [GPU-5](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4_XS-3.84bpw.gguf)

By “frontier,” we mean that no other plotted model is both faster and more accurate.

96 GB: RTX Pro 6000

RTX PRO 6000, the GPU with the most memory, lets us show the full range of models tested.

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 116.11 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 108.01 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 105.44 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 101.11 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 90.42 3.84
Unsloth
A UD-IQ2_S 0.8633 114.81 2.49
B UD-Q2_K_XL 0.9572 106.95 2.82
C UD-IQ3_XXS 0.9359 100.51 3.14
D UD-IQ3_S 0.9555 95.53 3.47
E UD-Q3_K_XL 0.9760 90.88 3.80
F UD-IQ4_XS 0.9920 86.73 4.13
G UD-Q4_K_S 0.9877 82.21 4.46
H UD-Q4_K_M 0.9703 78.58 4.79
I UD-Q4_K_XL 0.9871 74.75 5.12
J UD-Q5_K_S 0.9878 71.00 5.44
K UD-Q5_K_M 0.9897 67.71 5.77
L UD-Q5_K_XL 0.9905 65.52 6.10
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 112.77 2.50
b GSQ-RCO-IQ2_S 0.9364 108.22 2.75
c GSQ-RCO-IQ3_XXS 0.9438 103.39 3.00
d GSQ-RCO-IQ3_S 0.9943 94.77 3.50
Bartowski
a IQ2_XXS 0.7986 112.21 2.72
b IQ2_S 0.9296 106.33 2.99
c Q2_K 0.9616 96.08 3.45
d IQ3_XXS 0.9594 92.43 3.68
e IQ3_XS 0.9582 88.36 3.89
f IQ3_M 0.9667 86.17 4.06
AtomicChat
a AD-IQ2_XXS 0.8385 116.28 2.58
b AD-IQ2_XS 0.9296 108.67 2.85
c AD-IQ2_S 0.9061 100.28 3.22
d AD-IQ3_XXS 0.9644 95.29 3.50
e AD-IQ3_S 0.9730 88.52 4.04

GPU-5

32 GB: RTX 5090

The RTX 5090 tells a similar story, leading to the same recommendations.

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 119.13 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 110.78 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 108.08 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 103.57 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 93.66 3.84
Unsloth
A UD-IQ2_S 0.8633 115.34 2.49
B UD-Q2_K_XL 0.9572 108.12 2.82
C UD-IQ3_XXS 0.9359 102.87 3.14
D UD-IQ3_S 0.9555 97.87 3.47
E UD-Q3_K_XL 0.9760 93.46 3.80
F UD-IQ4_XS 0.9920 89.69 4.13
G UD-Q4_K_S 0.9877 85.28 4.46
H UD-Q4_K_M 0.9703 81.63 4.79
I UD-Q4_K_XL 0.9871 77.51 5.12
J UD-Q5_K_S 0.9878 73.60 5.44
K UD-Q5_K_M 0.9897 69.87 5.77
L UD-Q5_K_XL 0.9905 67.59 6.10
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 114.11 2.50
b GSQ-RCO-IQ2_S 0.9364 110.19 2.75
c GSQ-RCO-IQ3_XXS 0.9438 105.67 3.00
d GSQ-RCO-IQ3_S 0.9943 95.49 3.50
Bartowski
a IQ2_XXS 0.7986 115.77 2.72
b IQ2_S 0.9296 109.39 2.99
c Q2_K 0.9616 99.29 3.45
d IQ3_XXS 0.9594 95.83 3.68
e IQ3_XS 0.9582 91.14 3.89
f IQ3_M 0.9667 89.08 4.06
AtomicChat
a AD-IQ2_XXS 0.8385 119.53 2.58
b AD-IQ2_XS 0.9296 112.39 2.85
c AD-IQ2_S 0.9061 103.46 3.22
d AD-IQ3_XXS 0.9644 98.22 3.50
e AD-IQ3_S 0.9730 91.72 4.04 
        Once again [GPU-5](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4_XS-3.84bpw.gguf)[GPU-4](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3_S-3.23bpw.gguf)

24 GB: RTX 4090 and RTX 3090

Both 24 GB cards fit all five ShapeLearn models. We plot them separately because their throughput differs, but the ordering is the same on both.

RTX 4090

        The RTX 4090 keeps the same pattern: [GPU-5](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4_XS-3.84bpw.gguf)

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 78.67 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 72.89 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 71.12 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 67.77 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 59.16 3.84
Unsloth
A UD-IQ2_S 0.8633 78.23 2.49
B UD-Q2_K_XL 0.9572 72.79 2.82
C UD-IQ3_XXS 0.9359 67.24 3.14
D UD-IQ3_S 0.9555 63.02 3.47
E UD-Q3_K_XL 0.9760 59.13 3.80
F UD-IQ4_XS 0.9920 55.62 4.13
G UD-Q4_K_S 0.9877 52.44 4.46
H UD-Q4_K_M 0.9703 49.77 4.79
I UD-Q4_K_XL 0.9871 47.08 5.12
J UD-Q5_K_S 0.9878 44.80 5.44
K UD-Q5_K_M 0.9897 42.45 5.77
L UD-Q5_K_XL 0.9905 40.84 6.10
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 77.80 2.50
b GSQ-RCO-IQ2_S 0.9364 73.34 2.75
c GSQ-RCO-IQ3_XXS 0.9438 69.26 3.00
d GSQ-RCO-IQ3_S 0.9943 62.41 3.50
Bartowski
a IQ2_XXS 0.7986 75.28 2.72
b IQ2_S 0.9296 71.27 2.99
c Q2_K 0.9616 63.74 3.45
d IQ3_XXS 0.9594 61.03 3.68
e IQ3_XS 0.9582 58.20 3.89
f IQ3_M 0.9667 56.16 4.06
AtomicChat
a AD-IQ2_XXS 0.8385 79.51 2.58
b AD-IQ2_XS 0.9296 74.19 2.85
c AD-IQ2_S 0.9061 67.40 3.22
d AD-IQ3_XXS 0.9644 63.60 3.50
e AD-IQ3_S 0.9730 57.38 4.04

RTX 3090

Older, but still fast in these measurements.

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 53.03 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 51.20 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 50.75 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 49.49 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 45.79 3.84
Unsloth
A UD-IQ2_S 0.8633 52.82 2.49
B UD-Q2_K_XL 0.9572 50.37 2.82
C UD-IQ3_XXS 0.9359 48.16 3.14
D UD-IQ3_S 0.9555 46.39 3.47
E UD-Q3_K_XL 0.9760 46.28 3.80
F UD-IQ4_XS 0.9920 46.12 4.13
G UD-Q4_K_S 0.9877 44.49 4.46
H UD-Q4_K_M 0.9703 43.30 4.79
I UD-Q4_K_XL 0.9871 41.37 5.12
J UD-Q5_K_S 0.9878 39.37 5.44
K UD-Q5_K_M 0.9897 37.38 5.77
L UD-Q5_K_XL 0.9905 36.16 6.10
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 51.56 2.50
b GSQ-RCO-IQ2_S 0.9364 50.07 2.75
c GSQ-RCO-IQ3_XXS 0.9438 48.66 3.00
d GSQ-RCO-IQ3_S 0.9943 47.19 3.50
Bartowski
a IQ2_XXS 0.7986 53.63 2.72
b IQ2_S 0.9296 51.21 2.99
c Q2_K 0.9616 45.34 3.45
d IQ3_XXS 0.9594 47.46 3.68
e IQ3_XS 0.9582 44.68 3.89
f IQ3_M 0.9667 43.42 4.06
AtomicChat
a AD-IQ2_XXS 0.8385 53.95 2.58
b AD-IQ2_XS 0.9296 51.91 2.85
c AD-IQ2_S 0.9061 48.24 3.22
d AD-IQ3_XXS 0.9644 47.10 3.50
e AD-IQ3_S 0.9730 47.22 4.04

GPU-4GPU-5GPU-5

16 GB: RTX 4080 and RTX 5060 Ti

With a tighter VRAM budget, the pragmatic choice is to leave room for the context you need, not just the model weights. These plots contain fewer competing configurations, but all five ShapeLearn models are represented.

RTX 4080

        On the RTX 4080, [GPU-4](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ3_S-3.23bpw.gguf)[GPU-5](https://huggingface.co/byteshape/Qwen3.8-27B-GGUF/blob/main/Qwen3.8-27B-IQ4_XS-3.84bpw.gguf)

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 62.01 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 56.84 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 55.14 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 52.43 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 45.74 3.84
Unsloth
A UD-IQ2_S 0.8633 61.70 2.49
B UD-Q2_K_XL 0.9572 56.70 2.82
C UD-IQ3_XXS 0.9359 52.32 3.14
D UD-IQ3_S 0.9555 49.06 3.47
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 60.47 2.50
b GSQ-RCO-IQ2_S 0.9364 57.37 2.75
c GSQ-RCO-IQ3_XXS 0.9438 54.19 3.00
d GSQ-RCO-IQ3_S 0.9943 48.42 3.50
Bartowski
a IQ2_XXS 0.7986 58.69 2.72
b IQ2_S 0.9296 55.39 2.99
c Q2_K 0.9616 48.91 3.45
d IQ3_XXS 0.9594 46.99 3.68
e IQ3_XS 0.9582 44.71 3.89
AtomicChat
a AD-IQ2_XXS 0.8385 62.55 2.58
b AD-IQ2_XS 0.9296 57.82 2.85
c AD-IQ2_S 0.9061 52.43 3.22
d AD-IQ3_XXS 0.9644 49.15 3.50

RTX 5060 Ti

On the RTX 5060 Ti, the corresponding figures are 33.1 tok/s and 29.1 tok/s.

Show Legend #

# Model Acc TPS BPW
ByteShape
GPU-1 IQ2_XXS-2.56bpw 0.9304 38.05 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 35.41 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 34.50 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 33.05 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 29.15 3.84
Unsloth
A UD-IQ2_S 0.8633 38.02 2.49
B UD-Q2_K_XL 0.9572 35.42 2.82
C UD-IQ3_XXS 0.9359 32.80 3.14
D UD-IQ3_S 0.9555 30.97 3.47
ISTA-DASLab
a GSQ-RCO-IQ2_XS 0.8647 37.62 2.50
b GSQ-RCO-IQ2_S 0.9364 35.80 2.75
c GSQ-RCO-IQ3_XXS 0.9438 33.95 3.00
d GSQ-RCO-IQ3_S 0.9943 30.72 3.50
Bartowski
a IQ2_XXS 0.7986 36.33 2.72
b IQ2_S 0.9296 34.76 2.99
c Q2_K 0.9616 30.79 3.45
d IQ3_XXS 0.9594 29.69 3.68
e IQ3_XS 0.9582 28.24 3.89
AtomicChat
a AD-IQ2_XXS 0.8385 38.42 2.58
b AD-IQ2_XS 0.9296 36.05 2.85
c AD-IQ2_S 0.9061 32.61 3.22
d AD-IQ3_XXS 0.9644 31.01 3.50

GPU-5GPU-4

ShapeLearn-Lite, in retrospect #

ShapeLearn-Lite uses a smaller optimization budget than full ShapeLearn. It let us get Qwen 3.8 27B onto 12 GB to 24 GB GPUs within a few days.

We released after targeted sanity checks and started the full evaluation afterwards. The full ShapeLearn models were ready before the benchmarking was finished. Evaluating both sets, along with the competing models, is what took most of the time.

Then Unsloth released its Dynamic v3 models. At similar sizes, several had lower KLD than Lite in our measurements. On KLD alone, Lite looked less competitive.

KLD looked decisive

KLD measures divergence between a quantized model’s predicted token distributions and the BF16 reference under a particular evaluation setup. It is useful for diagnosing substantial changes, but lower divergence does not automatically mean better task performance.

We measure KLD on a dataset of about 5 million tokens of prompt and response pairs, drawn from several benchmarks, including long-context and agentic tasks. We also changed how KLD is computed, so that it is closer to what we expect KLD to measure:

  • KLD is measured on response tokens only, not on prompt tokens. We do not want to measure how well a model can generate prompts.
  • KLD only considers the tokens that have a chance of being sampled during generation, the top-20, top-40, or top-60 tokens at each position. The tail tokens never get sampled, so they do not contribute.
  • Requests have clear boundaries. Each prompt and response pair is scored as its own request, not as part of one long concatenated stream.

Show Legend #

# Model KLD Size (GB) BPW
ShapeLearn-Lite
Lite-1 IQ3_S-3.44bpw 0.035875 10.79 3.44
Lite-2 IQ4_XS-3.67bpw 0.028296 11.51 3.68
Lite-3 IQ4_XS-4.00bpw 0.018249 12.52 4.00
Lite-4 IQ4_XS-4.40bpw 0.009901 13.78 4.40
Lite-5 Q5_K_S-4.72bpw 0.007578 14.78 4.72
Lite-6 Q5_K_M-5.60bpw 0.003297 17.53 5.60
Unsloth
i UD-IQ1_S 0.389550 5.76 1.84
ii UD-IQ1_M 0.261876 6.26 2.00
iii UD-IQ2_XXS 0.181493 6.76 2.16
iv UD-IQ2_S 0.108374 7.79 2.49
v UD-Q2_K_XL 0.065200 8.81 2.81
vi UD-IQ3_XXS 0.040407 9.84 3.14
vii UD-IQ3_S 0.028759 10.87 3.47
viii UD-Q3_K_XL 0.019844 11.90 3.80
ix UD-IQ4_XS 0.011992 12.93 4.13
x UD-Q4_K_S 0.008827 13.96 4.46
xi Q4_0 0.019264 14.69 4.69
xii UD-Q4_K_M 0.007054 14.99 4.79
xiii UD-Q4_K_XL 0.005210 16.01 5.11
xiv Q4_1 0.009603 16.06 5.13
xv UD-Q5_K_S 0.003619 17.04 5.44
xvi UD-Q5_K_M 0.002839 18.07 5.77
xvii UD-Q5_K_XL 0.002432 19.10 6.10
xviii UD-Q6_K 0.001771 20.13 6.43
xix UD-Q6_K_M 0.001426 21.16 6.76
xx UD-Q6_K_L 0.001150 22.19 7.09
xxi UD-Q6_K_XL 0.000985 23.22 7.42
xxii UD-Q8_K_L 0.000726 25.78 8.23
xxiii Q8_0 0.000648 26.62 8.50
xxiv UD-Q8_K_XL 0.000503 28.76 9.19

For example, Unsloth’s UD-IQ3_S (vii) has about 20% lower KLD than the similarly sized smallest Lite model (Lite-1): 0.028759 versus 0.035875. Yet its aggregate benchmark score is lower: 95.55% versus 97.33% of BF16.

If lower KLD were sufficient to rank these models by task performance, the benchmark ordering should have followed it.

It did not.

        The point is not that KLD is useless. It is that a fidelity ranking is not a task-performance ranking. This is the distinction explored in our [KLD evaluation blog](https://byteshape.com/blogs/Evaluating-Quantized-Models/part-2/). Our [related paper on KLD and quantization fidelity metrics](https://arxiv.org/abs/2606.19558) was also recently accepted to the [EMNLP](https://2026.emnlp.org/) Industry Track.

Lite held up

Naturally, we made more plots.

Here, we show the RTX Pro 6000 because it can accommodate the full comparison. Each model’s benchmark score is reused across the GPU plots; the measured throughput and the set of displayed models change.

Show Legend #

# Model Acc TPS BPW
ShapeLearn (this release)
GPU-1 IQ2_XXS-2.56bpw 0.9304 116.11 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 108.01 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 105.44 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 101.11 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 90.42 3.84
ShapeLearn-Lite
Lite-1 IQ3_S-3.44bpw 0.9733 98.74 3.45
Lite-2 IQ4_XS-3.67bpw 0.9802 94.48 3.68
Lite-3 IQ4_XS-4.00bpw 0.9880 89.85 4.00
Lite-4 IQ4_XS-4.40bpw 0.9856 85.03 4.40
Lite-5 Q5_K_S-4.72bpw 0.9909 79.75 4.72
Lite-6 Q5_K_M-5.60bpw 0.9919 70.28 5.60
Unsloth
A UD-IQ2_S 0.8633 114.81 2.49
B UD-Q2_K_XL 0.9572 106.95 2.82
C UD-IQ3_XXS 0.9359 100.51 3.14
D UD-IQ3_S 0.9555 95.53 3.47
E UD-Q3_K_XL 0.9760 90.88 3.80
F UD-IQ4_XS 0.9920 86.73 4.13
G UD-Q4_K_S 0.9877 82.21 4.46
H UD-Q4_K_M 0.9703 78.58 4.79
I UD-Q4_K_XL 0.9871 74.75 5.12
J UD-Q5_K_S 0.9878 71.00 5.44
K UD-Q5_K_M 0.9897 67.71 5.77
L UD-Q5_K_XL 0.9905 65.52 6.10

Leaving the full ShapeLearn models aside for a moment, three of the six ShapeLearn-Lite models sit on the Lite-versus-Unsloth frontier: the three smallest Lite models, the lighter orange bubbles labelled 1-3.

Of the twelve Unsloth v3 models shown, three also sit on that frontier: UD-IQ2_S (A), UD-Q2_K_XL (B), and UD-IQ4_XS (F). UD-IQ4_XS (F) is a strong higher-quality point, while Lite earns its places in the middle of the range.

Add the five full ShapeLearn models back in (the darker orange bubbles), and they take over the entire frontier.

Lite was never meant to be the final result. It still held its own where it mattered.

Speculative Decoding #

We also evaluated MTP and DFlash2 with the new models, using 3 draft tokens for MTP and 7 draft tokens for DFlash2. Both methods increased throughput for all five ShapeLearn models on all six GPUs tested.

DFlash2 was faster than MTP in almost all cases. Across the full lineup, DFlash2 reached 1.34-2.10x the baseline next-token prediction (NTP) throughput, while MTP reached 1.28-1.66x.

We measured with the sampling parameters Qwen recommends for thinking mode, over a diverse set of agentic coding, mathematics, and general-knowledge requests. The speedups would likely be larger under greedy decoding, but temperature-based sampling better reflects real usage.

The figure below shows NTP, MTP, and DFlash2 throughput for each GPU. The quality axis is the target-model benchmark score reported above. These plots do not independently establish quality equivalence between decoding methods.

Show Legend #

# Model Acc NTP TPS MTP TPS DFlash2 TPS BPW
RTX Pro 6000 (96 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 116.11 165.52 (1.43x) 172.01 (1.48x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 108.01 153.48 (1.42x) 165.94 (1.54x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 105.44 152.19 (1.44x) 166.03 (1.57x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 101.11 146.85 (1.45x) 164.24 (1.62x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 90.42 145.69 (1.61x) 150.83 (1.67x) 3.84
RTX 5090 (32 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 119.13 164.44 (1.38x) 175.53 (1.47x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 110.78 156.63 (1.41x) 175.97 (1.59x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 108.08 155.19 (1.44x) 171.97 (1.59x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 103.57 148.34 (1.43x) 169.69 (1.64x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 93.66 147.00 (1.57x) 166.57 (1.78x) 3.84
RTX 4090 (24 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 78.67 111.73 (1.42x) 135.39 (1.72x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 72.89 108.40 (1.49x) 132.20 (1.81x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 71.12 107.21 (1.51x) 133.38 (1.88x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 67.77 101.47 (1.50x) 131.12 (1.93x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 59.16 98.25 (1.66x) 124.15 (2.10x) 3.84
RTX 3090 (24 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 53.03 68.19 (1.29x) 70.94 (1.34x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 51.20 65.76 (1.28x) 68.90 (1.35x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 50.75 65.82 (1.30x) 68.40 (1.35x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 49.49 64.50 (1.30x) 66.34 (1.34x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 45.79 66.55 (1.45x) 63.93 (1.40x) 3.84
RTX 4080 (16 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 62.01 87.06 (1.40x) 101.63 (1.64x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 56.84 81.25 (1.43x) 96.90 (1.70x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 55.14 79.74 (1.45x) 96.95 (1.76x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 52.43 76.88 (1.47x) 94.09 (1.79x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 45.74 74.01 (1.62x) 87.49 (1.91x) 3.84
RTX 5060 Ti (16 GB)
GPU-1 IQ2_XXS-2.56bpw 0.9304 38.05 50.54 (1.33x) 56.71 (1.49x) 2.56
GPU-2 IQ3_XXS-2.88bpw 0.9656 35.41 47.94 (1.35x) 52.57 (1.48x) 2.88
GPU-3 IQ3_XS-3.01bpw 0.9726 34.50 47.93 (1.39x) 52.28 (1.52x) 3.01
GPU-4 IQ3_S-3.23bpw 0.9872 33.05 46.25 (1.40x) 50.34 (1.52x) 3.23
GPU-5 IQ4_XS-3.84bpw 0.9963 29.15 45.82 (1.57x) 47.01 (1.61x) 3.84

There is also a memory tradeoff between the two approaches. The embedded quantized MTP weights add only about 250 MB to the model, and if MTP is not used, these weights are not loaded into GPU memory. In comparison, the 4-bit DFlash2 draft model is about 1.1 GB, so enabling DFlash2 requires roughly 1.1 GB of additional GPU memory.

Packaging MTP as a separate GGUF file would largely eliminate this advantage. The standalone model would need its own MTP embedding and output layers, which are by far its largest tensors, bringing its memory footprint to roughly 1 GB as well.

In addition, DFlash2 in llama.cpp currently does not support image inputs, which is an important consideration for multimodal use cases.

Benchmarking Methodology #

We evaluate all reported models across a set of instruct and thinking benchmarks.

Instruct benchmarks:

  • GSM8K for math
  • IFEval for instruction following
  • MMLU for general knowledge
  • LiveCodeBench V6* for coding
  • Multi-IF for multi-turn and multilingual instruction following
  • ACEBench for tool use and agentic tasks

Thinking benchmarks:

  • ACEBench for tool use and agentic tasks
  • Multiple HumanEval for coding
  • BFCL V4* for tool calling and agentic tasks

For the thinking benchmarks, we used Qwen 3.8’s medium thinking setting.

For each benchmark, the score of a quantized model is normalized by the score of the corresponding BF16 model. The overall reported score is the average of these normalized benchmark scores.

Our LiveCodeBench V6* evaluation includes problems from January 1, 2024 onward, excluding the 2023 problems. We found the 2023 problems to be relatively easy for current models, with most models achieving very high scores on them. As a result, they provide limited discrimination between models while adding substantial evaluation time.

For BFCL V4*, we evaluate the following eight subsets:

  • live_simple
  • live_parallel
  • live_parallel_multiple
  • live_relevance
  • multi_turn_base
  • multi_turn_miss_func
  • multi_turn_miss_param
  • multi_turn_long_context

All evaluations were run with llama.cpp b10430. For both instruct and thinking experiments, we use the sampling parameters recommended by Qwen for the corresponding mode.

Conclusion #

ShapeLearn-Lite did what it was designed to do. It got useful Qwen 3.8 27B quants onto 12 to 24 GB GPUs quickly, and it held up better than its KLD ranking suggested.

Full ShapeLearn goes further. It improves the measured quality-speed trade-offs over Lite and contributes five frontier models across all six tested GPUs.

GPU-5GPU-4

KLD remains useful, but it is not a task-performance leaderboard. Fidelity metrics tell us how much the model’s distributions changed under a particular measurement. Benchmarks tell us whether those changes matter on the tasks we tested.

We were impatient. This time, it worked out pretty well.

source & further reading
byteshape.com — original article
── more in #ai-tools 4 stories · sorted by recency
── more on @byteshape 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/shapelearn-qwen-3-8-…] indexed:0 read:26min 2026-09-18 · —