VBR k/v cache is actually usable (buun-llama)

So i have been running qwen-3.8-flash-next iq3 with great results with just 64gb of ram, a 9950x3d, a 9070xt and a zfs znvme stripe of 3 mid range nvmes.

the vbr that buun uses fits 255k of context on this setup and i am getting ~ 120 pp and ~ 30 of token gen.

this is the actual command

llama-server \

-m Qwen3.8-Flash-*00001*.gguf \

-c $((255* 1024)) \

-fa on -ctk vbr -ctv vbr --vbr-min-bits t4 --moe-cache soft \

–host 0.0.0.0 --port 11434 --reasoning on --reasoning-effort medium --reasoning-preserve --metrics \

–spec-type draft-mtp --model-draft ../mtp-Qwen3.8-Flash-Next-shared-Q8_0.gguf \

-b 512 -ub 512 --fit on --slot-prompt-similarity 0.2

i have not really gone and optimized the b /ub and moe cache but yea with qwen-code it achieved several re tasks, quite fast too.