VBR k/v cache is actually usable (buun-llama)

A user running Qwen3.8-Flash-Next at IQ3 quantization on a system with 64GB of RAM, an AMD 9950X3D, a 9070XT, and a ZFS stripe of three mid-range NVMe drives reported fitting 255K tokens of context using buun-llama's VBR k/v cache, achieving roughly 120 prompt-processing tokens per second and about 30 tokens per second of generation. The configuration used the flags -ctk vbr -ctv vbr --vbr-min-bits t4 --moe-cache soft with a draft-mtp speculative model, and the user said the setup completed several reasoning tasks quickly via qwen-code without tuning batch or MoE cache settings.

So i have been running qwen-3.8-flash-next iq3 with great results with just 64gb of ram, a 9950x3d, a 9070xt and a zfs znvme stripe of 3 mid range nvmes. the vbr that buun uses fits 255k of context on this setup and i am getting ~ 120 pp and ~ 30 of token gen. this is the actual command llama-server \ -m Qwen3.8-Flash- 00001 .gguf \ -c $ 255 1024 \ -fa on -ctk vbr -ctv vbr --vbr-min-bits t4 --moe-cache soft \ –host 0.0.0.0 --port 11434 --reasoning on --reasoning-effort medium --reasoning-preserve --metrics \ –spec-type draft-mtp --model-draft ../mtp-Qwen3.8-Flash-Next-shared-Q8 0.gguf \ -b 512 -ub 512 --fit on --slot-prompt-similarity 0.2 i have not really gone and optimized the b /ub and moe cache but yea with qwen-code it achieved several re tasks, quite fast too.