# Qwen3.8-Flash-Next UD-Q4_K_XL on one DGX Spark: 128K ctx + q8 KV (fix for qwen4exp self_k_rot assert)

> Source: <https://gist.github.com/CocaKova/3cbfbf4991e93eee39f4c7612712e487>
> Published: 2026-08-26 21:45:46+00:00

Tested 2026-08-26 on a DGX Spark (GB10, 128 GB unified). Measured, not estimated.

With `-ctk q8_0 -ctv q8_0`

, llama.cpp PR #27742 (`qwen4exp`

) dies at load:

``` php
src/models/qwen4exp.cpp:544: GGML_ASSERT(inp->self_k_rot == nullptr && inp->self_v_rot == nullptr) failed
```

Cause: a quantized KV cache auto-enables llama.cpp's Hadamard "attention rotation" on K/V, and the qwen4exp QSA sparse-attention path doesn't support rotated caches yet.

**Fix: disable the rotation with an env var.**

```
LLAMA_ATTN_ROT_DISABLE=1
```

- llama.cpp: PR #27742 (
`qwen4exp`

), commit`035e22731`

, CUDA build - Weights:
`unsloth/Qwen3.8-Flash-Next-GGUF`

→`UD-Q4_K_XL/`

(4 shards, 103.7 GiB) - Optional vision:
`DevQuasar/Qwen.Qwen3.8-Flash-Next-GGUF`

→`mmproj-…-f16.gguf`

```
LLAMA_ATTN_ROT_DISABLE=1 ./build/bin/llama-server \
  -m UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf \
  --mmproj mmproj-Qwen.Qwen3.8-Flash-Next.f16.gguf \
  --host 0.0.0.0 --port 8000 \
  -ngl 999 -c 131072 -np 1 -fa on --jinja -t 20 \
  -ctk q8_0 -ctv q8_0 \
  --load-mode mmap --override-tensor 'per_layer_token_embd=CPU' \
  --reasoning-format auto
```

Notes:

`--override-tensor 'per_layer_token_embd=CPU'`

+`--load-mode mmap`

keeps the 51B n-gram / PLE table mmap'd from SSD instead of resident. On unified memory that's the only "offload" that actually frees anything — host-offload flags are meaningless when host RAM == GPU RAM.- Load takes ~5 minutes. You'll see one
`NVRM: ... Out of memory [NV_ERR_NO_MEMORY]`

line in dmesg during the mmap page-in. It's harmless; the server comes up anyway. `-np 1`

: one 128K slot. Only the full-attention layers have a KV cache to quantize — the GDN/linear layers keep recurrent state regardless, so KV is cheap either way.- Free memory first: everything else on the box has to be down (other models, TTS, embed servers). Baseline here was ~11 GB used before launch.

| Idle after load | 89 GB used / 32 GB available |
| 30,183-token prompt | prefill 414 tok/s (73 s), gen 15.7 tok/s |
| Peak during that prompt | 91 GB used / 30 GB spare |
| Short prompt gen | ~18.5 tok/s |

Output coherent; thinking on/off via `chat_template_kwargs.enable_thinking`

works; `reasoning_content`

populated.