{"slug": "qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths", "title": "Qwen3.8-27B on a single RTX 3090: crash fix, 131K context, 9 myths", "summary": "A 14-hour benchmark on a single NVIDIA GeForce RTX 3090 found that the Qwen3.8-27B hybrid SSM+attention model achieves a 131K context window on 24 GB VRAM, scoring 20/21 on a frontier test set, but crashes on any image input due to a CUDA error that is fixed by setting GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32. The test, run by an unnamed developer using llama.cpp build b10217-ddd4ec142, also debunks 9 myths about the model and settings.", "body_md": "Roughly 14 hours of measurements on a single consumer card. Every answer was computed by a reference implementation before the model was asked. Every wrong turn is kept in section 13 rather than quietly edited out.\n\n| Component | Version / spec |\n|---|---|\n| OS | Debian GNU/Linux 13 (trixie), kernel 6.12.101+deb13-amd64 |\n| Desktop | KDE Plasma on Wayland (kwin_wayland) |\n| CPU | AMD Ryzen 9 5950X — 16C/32T, Zen 3 |\n| RAM | 31 GiB |\n| GPU | NVIDIA GeForce RTX 3090, 24576 MiB (GA102, sm_86) |\n| Driver | 610.57.04 · CUDA UMD 13.3 |\n| PCIe | gen 4 x16 under load (gen 2 at idle is power saving, not a fault) |\n| Power limit | 380 W (VBIOS default 370, max 380), pinned via systemd |\n| Storage | Samsung SSD 970 PRO 1TB NVMe |\n| Inference | llama.cpp b10217-ddd4ec142 (llama.app installer) |\n| Model A | unsloth/Qwen3.8-27B-GGUF · UD-Q4_K_XL · 17,923,394,624 B · rev f1bfb127 |\n| Model B | AtomicChat/Qwen3.8-27B-GGUF · AD-Q4_K_M · 17,120,781,792 B |\n| Vision proj. | mmproj-F16.gguf · 927,607,488 B |\n| Harnesses | opencode 1.18.18 · Hermes v0.20.2 (2026.8.16) · atomic-agent v0.2.1 * |\n\n* atomic-agent does not self-report a version. v0.2.1 is inferred from the latest\nGitHub release of `AtomicBot-ai/atomic-agent`\n\n(2026-08-14) at install time (2026-08-16).\n\nThe GGUF reports `general.architecture = qwen35`\n\n, and it is a **hybrid SSM + attention**\nmodel, not a plain transformer. Only one layer in four keeps a KV cache. That single fact is why a 27B model\ngets a 131K window on a 24 GB card.\n\n```\nblock_count              65        full_attention_interval  4     ← hybrid\nattention.head_count     24        attention.head_count_kv  4\nattention.key_length    256        attention.value_length 256\nembedding_length       5120        context_length      262144\nnextn_predict_layers      1        ← MTP heads ship with the model\ngeneral.sampling      temp 1.0 · top_k 20 · top_p 0.95\n```\n\nMost \"which model / which setting\" advice in circulation is eyeballed — side-by-side screenshots judged by\ntaste — or copied from someone running different hardware. This benchmark only asks questions with a\n**verifiable answer**:\n\n**What is deliberately not measured:** anything judged by taste. Web-design quality, prose\nstyle, \"which output looks nicer\". Those matter, but a benchmark that scores them is reporting the author's\npreferences with extra steps.\n\nThe first problem sets were too easy — everything scored 100%, which measures the test, not the model. Difficulty was raised three times. The progression is itself a result:\n\n| Set | Character | Score |\n|---|---|---|\n| Easy — 8 short problems | arithmetic, counting | 8/8 |\n| Hard — 5 multi-step | simulation, search, DP | 5/5 with reasoning |\n| Brutal — 5 heavy | 200-term recurrence, 8×8 grid DP | 4/5 |\n| Frontier — 7 categories × 3 reps | traps, bug-hunting, knapsack, probability | 20/21 |\n\nTelemetry depth is not uniform. A single long-lived `nvidia-smi`\n\nprocess\nsampled temperature, power, clocks and utilisation every 2 s with native timestamps, at **0.0% measured\nCPU overhead** — no per-sample process spawning, and nothing called inside the timed path. Later probes\nrecord start/end timestamps per row and are joined against that log offline. Earlier probes carry wall-clock\nand token counts only.\n\nAny request containing an image killed the server instantly. Text-only requests ran indefinitely.\n\n```\nggml-cuda.cu:106: CUDA error\nCUDA error: the requested functionality is not supported\n  current device: 0, in function ggml_cuda_mul_mat_cublas_impl at ...:1548\n  cublasGemmEx(..., cu_data_type_a, ..., cu_data_type_b, ..., cu_compute_type,\n               CUBLAS_GEMM_DEFAULT_TENSOR_OP)\n```\n\n`-c 16384`\n\nand `-c 131072`\n\n, ~2 GB free at\nabort. The status is `NOT_SUPPORTED`\n\n, not OOM.`--image-min-tokens`\n\n, with\n`--image-max-tokens 512`\n\n, at 640×480 and 1920×1080.\n\n```\nexport GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32\n```\n\n| Value | 1920×1080 screenshot | Latency |\n|---|---|---|\n| fp32 | works — reads the error text verbatim, with its line number | 3.4 s |\n| fp16 | abort | — |\n| unset | abort | — |\n\n** fp16 does not help; only fp32 does.** Measured cost: none.\n66.11 tok/s with it against 65.28 without, same seed, thermally equalised. Prefill over a 128,290-token\nprompt was unchanged at 705 tok/s.\n\nThe CPU fallback (`--no-mmproj-offload`\n\n) also avoids the abort and is unusable: 28.6 s at 640px\n— *and the model hallucinated the contents*, because the text is illegible at that scale — rising to\nover 300 s at 1080p, against 3.4 s on GPU.\n\n**This is a workaround, not a fix.** It forces fp32 for *every* cuBLAS matmul,\nnot just the one that breaks. Upstream:\n[ggml-org/llama.cpp#24999](https://github.com/ggml-org/llama.cpp/issues/24999).\n\nWith q4_0 K+V the **measured** cost is **28,672 bytes per token**:\n\n```\nVRAM_llama = weights + 28,672 B × context + 185 MiB\n             (+ 1,125 MiB when the vision projector sits on the GPU)\n```\n\nThat reproduces six independent boot measurements to within 25 MiB.\n\n`full_attention_interval`\n\ngives 18,432 — llama.cpp allocates KV for roughly 25 of the 65 layers,\nnot 16. If you compute a context budget from GGUF metadata, verify it by booting.Ceilings were verified by *filling* them: a 128,290-token prompt with a needle hidden at the start,\nrecovered verbatim. 262,144 OOMs. 163,840 boots and runs with ~1 GiB free.\n\nA widely-shared claim says lowering `-c`\n\nbuys speed. Tested directly with a short prompt at every\nsetting, which isolates allocation from fill:\n\nOnly VRAM grows with -c. Throughput moves 1.2% between 8K and 131K.\n\n**Keep the window.** The reported speedups from shrinking `-c`\n\ncome from processing\nless context, not from allocating less. Filling it is what costs: the same prompt at 53,054 tokens of context\nmade the model think 63% more and take nearly twice as long — while staying **4/4 correct**.\nAccuracy holds at a full context; latency does not.\n\nThe model ships MTP heads, so `--spec-type draft-mtp`\n\nworks with no extra download. It is\n**off by default**, and the wrong spelling fails loudly — `--spec-type mtp`\n\nrefuses to\nstart with `unknown speculative type`\n\n, so nobody is silently running without it because of a typo.\nThey are running without it because they never set the flag.\n\nLonger drafts raise mean accepted length but acceptance falls faster, and every rejected token is wasted verification compute.\n\n**n=2 wins on this GPU.** A widely-cited figure puts the sweet spot at n=3 — that measurement\nwas taken on an RTX 6000. The optimum depends on spare verification compute, which is a property of the card.\nMTP itself is the big win: **+77%**. Its VRAM cost is **834 MiB**, not the 2–3 GB\nsometimes quoted.\n\nAcceptance counts parsed from raw accepted/generated totals. One shared scale — acceptance is plotted ×100 so both series share an axis honestly.\n\n| temp | tok/s | accepted | generated | acceptance | without MTP |\n|---|---|---|---|---|---|\n| 0.0 | 68.73 | 502 | 590 | 0.851 | 38.91 |\n| 0.6 | 69.47 | 506 | 582 | 0.869 | 39.25 |\n| 0.7 | 68.67 | 506 | 582 | 0.869 | 39.33 |\n| 1.0 | 63.00 | 476 | 642 | 0.741 | 39.31 |\n| 1.3 | 67.03 | 496 | 602 | 0.824 | 38.45 |\n\nThe last column is the control: **without MTP, temperature does nothing** (38.9 → 39.3 → 38.5,\nflat within noise). That localises the entire effect in the speculative path. At temp 1.0 acceptance drops to\n0.741 and throughput follows, ~9% down from 0.6.\n\nThe model card recommends temp 1.0. On this hardware **0.6–0.7 gives ~10% more throughput**,\nand accuracy was identical (8/10 at both) on verifiable problems, and on code generation (99% at both 0.6 and\n1.0). *Scope: single-answer arithmetic and code. Temperature very plausibly matters for open-ended work,\nwhich this benchmark does not measure.*\n\n** --spec-draft-p-min did not reproduce.** A discussion reports 15%\nfaster. Sweeping 0.00 → 0.50 at two temperatures moved throughput by at most 2.4–3.3%, inside or barely\noutside noise, with acceptance unchanged.\n\nTwo Q4 GGUFs of the same model from two publishers who each claim their dynamic quantisation is better.\nPerplexity on identical corpora, 60 chunks, `-c 4096`\n\n, f16 KV.\n\n| Unsloth UD-Q4_K_XL | AtomicChat AD-Q4_K_M | |\n|---|---|---|\n| Perplexity — wikitext-2 | 5.8031 ±0.0385 | 5.8009 ±0.0384 |\nPerplexity — real code (138 files) | 2.1266 ±0.0105 | 2.1258 ±0.0105 |\n| File size | 17,093 MiB | 16,328 MiB |\n| Sustained decode @ 79 °C | 65.28 tok/s | 64.10 tok/s |\n\nOn general text the gap is **17× smaller than the error bar**; on code, **13×\nsmaller**. In quality terms these are the same model.\n\nBoth keep roughly 1.4 GiB free with a real desktop session open. The only variable is the weight file.\n\nWhat is real is the **765 MiB** difference, worth about **28,000 tokens of\ncontext** on a 24 GB card — the whole reason the usable window goes from 98K to 131K. AtomicChat wins\non size, not on speed: thermally equalised it is 1.8% *slower*.\n\nConventional wisdom says LLM decode is memory-bandwidth bound. Not here: memory clock stayed pinned at 9501 MHz across every power level, so the only variable was compute — and throughput tracked it.\n\nFive minutes of continuous load per level. Temperature is the steady-state average over the last two minutes, not a burst reading.\n\nFitting throughput against SM clock gives an exponent of **0.75** — 1.0 would be purely\ncompute-bound, 0.0 purely memory-bound. The hybrid SSM architecture and MTP verifying three tokens per step\nboth push it that way.\n\n79 °C sustained is well inside spec: this card throttles at 95 °C, its maximum operating temperature is\n93 °C, and the thermal-violation counter stayed at zero. Dropping to 300 W buys 74 °C for a **9.1%\nthroughput loss** — not worth it unless noise matters.\n\nThe real Linux undervolt is a positive clock offset, which shifts the V/F curve so the card runs faster at\nthe same voltage. It needs `nvidia-settings`\n\nwith **Coolbits** and a running\n**X server**. Under Wayland, nvidia-settings reads values but refuses writes, and as root it\ncannot reach the display at all. Nothing available without X substitutes: `-lgc`\n\nonly\n*caps* clocks, `--set-vf-derate`\n\napplies a *negative* offset, and forcing memory to\nits maximum with `-lmc 9751,9751`\n\n**reports success and does nothing** — the real\nclock stays at 9501 and throughput does not move.\n\n**PowerMizer: leave it on Normal.** Under load the GPU already sits at its highest\nperformance level; \"Prefer Maximum Performance\" only pins high clocks *at idle*. The card does run at\nPerformance Level 3 (memory 19002 MT/s) when Level 4 (19502) exists, but at 75% compute-bound that 2.6% of\nbandwidth is worth about 0.6% of throughput.\n\nThree separate mechanisms get conflated in most discussions. They compose, and two of them are invisible unless you read the chat template.\n\nThe template accepts exactly three values and injects a *different system instruction* for each:\n\n| Value | What the template injects | Score |\n|---|---|---|\n| xhigh (default) | \"think carefully, validate key assumptions, consider plausible alternatives, prioritise correctness\" | 5/5 |\n| low | \"keep your thinking brief and focused, moving directly to the conclusion\" | 4/5 |\n| medium | nothing — the if/elif covers only xhigh and low, so medium falls through with no instruction at all | 2/5 |\n\n** medium is the worst option**, and the template explains why: it is a valid value\nthat passes validation and then sets no guidance. Not \"medium effort\" —\n\n`\"reasoning_effort\": \"low\"`\n\nat the top level of the request is `chat_template_kwargs`\n\n, where an invalid value correctly raises a Jinja exception. Anyone using the\nstandard field believes they set `low`\n\nand is actually running `xhigh`\n\n.Advice in circulation puts the optimum at 6K–12K. Testing budgets against a fixed problem set cannot validate that: once a budget is large enough to let the model finish, every larger budget produces byte-identical output. The informative experiment is the opposite — hold the budget fixed and scale the problem until the model needs more than the cap allows.\n\nSame algorithm run for N steps. Blue is what the model spends unconstrained; orange is the same problems under a 2048-token cap. ✕ marks a wrong answer.\n\n**The break is at 60 steps** — exactly where natural thinking (2,954) first crosses the cap.\nBelow it, capped and uncapped runs are identical. Above it the capped model gives up and returns nonsense: 18,\n18 and 35 against true answers of 295, 2,666 and 1,932,916. Totals: no cap **4/5**, reasoning off\n2/5, cap 2048 **2/5**. **An insufficient budget is as bad as no reasoning at all.**\n\nLeft completely unbounded (`--reasoning-budget -1`\n\n, `max_tokens 100000`\n\n) it thought\n**25,772 tokens**, produced a complete 4,454-character answer, and closed with\n`finish_reason: stop`\n\nafter 552 s. It does not loop.\n\nThinking and the answer **share one context window** — there is no separate buffer. The KV\ncache holds prompt + thinking + answer as one sequence, and the answer is conditioned on the thinking by\nattending to it. `max_tokens`\n\ncovers both. Only *across turns* can thinking be dropped, which\nis what `preserve_thinking`\n\ncontrols.\n\n| Observed natural thinking | tokens |\n|---|---|\nc1 expression evaluator, xhigh — pathological, produced no code | 43,593 |\n| unbounded, coding prompt — healthy maximum | 25,772 |\n| c5 logic, xhigh | 22,793 |\n| 250-step ladder, no cap | 21,330 |\n| c4 Lawler, xhigh | 14,779 |\n\n**The model self-limits around 26,000 tokens.** A 100K budget would be harmless — never\nreached — but pointless, and the arithmetic is worth stating plainly: **100,000 thinking tokens at\n60 tok/s is 28 minutes for one answer**, and would consume 100K of a 131K window, leaving 31K for\nprompt plus answer. The useful range is 26K–32K: high enough never to truncate, below the point where the\nmodel starts to wander.\n\nContext envelope at `-c 131072`\n\n: a 2K prompt leaves 129K for thinking plus answer; a 50K prompt\nleaves 81K; a 90K prompt leaves 41K. Below about 90K of prompt, thinking never competes seriously with the\nanswer.\n\n**Context shift is disabled by default.** When the window fills, generation stops —\nit does not slide. Several \"no answer\" results in this benchmark are that ceiling biting, not the model\ngiving up.\n\nTwelve-turn agentic session where every turn required real reasoning:\n\nContext measured at turn 12; generated tokens summed across all 12 turns.\n\nPreserving is **40% faster and generates 41% fewer tokens**, for 69% more context. The\nmechanism is visible turn by turn: with preservation, thinking collapses to 25–47 tokens in the middle turns\nbecause the model can see its own earlier reasoning; without it, thinking stays at 150–470 as it re-derives\neach time. At turn 12 the cost is 2,227 tokens, projecting to ~7,400 at 40 turns — 5.6% of a 131K window.\n**Worth it.**\n\nFive tasks of escalating difficulty, generated directly by the model with no harness in the loop, scored by\nhidden test suites. Each suite was validated against a reference implementation first; `c2`\n\nand\n`c4`\n\nadditionally check random instances against exhaustive brute force.\n\n| Task | Why it is hard | Tests |\n|---|---|---|\n| c1 | Expression evaluator: truncation toward zero, repeated unary minus, malformed-input errors | 26 |\n| c2 | Job scheduling with cooldown — greedy fails | 19 |\n| c3 | Keys and doors — needs BFS over augmented state, naive BFS fails | 17 |\n| c4 | Lawler's algorithm — EDF, the obvious approach, fails with precedences; the correct order is built backwards from the latest deadline | 21 |\n| c5 | 11 bounded logical claims, decided by code that is actually executed. Includes Euler's polynomial (prime until n=40) and Mersenne (breaks at 2¹¹−1) | 11 |\n\nc1–c3 are solved perfectly without reasoning. c4 and c5 are the two where the model does not already have a free win.\n\n| Task group | reasoning off | effort xhigh | Verdict |\n|---|---|---|---|\n| c1–c3 (medium) | 124/124 · 9–11 s | 98/99 · up to 932 s | reasoning hurts |\n| c4 (Lawler) | 62/63 · 15–24 s | 63/63 · 153–304 s | reasoning helps |\n| c5 (logic) | 32/33 · 31–36 s | 33/33 · 454–518 s | reasoning helps |\n\n**Reasoning helps exactly at the edge of capability, and nowhere else.** Below that edge it is\npure cost. Above it — no case found. The price for the last 2–3% is 10–15× the wall time.\n\nA detail worth noting: on c4, `xhigh`\n\nproduced **48–54 lines** where reasoning-off\nproduced 86–127. Thinking led it to the compact correct algorithm rather than a longer construction.\n\n** xhigh on c1 is actively destructive.** Across nine attempts at three\ntemperatures,\n\nAn easy sweep (unambiguous requests, 5/12/19/30 tools available) scored **perfect at every level,\nincluding 30 tools** — zero wrong picks, zero malformed arguments. So a hard set was built around real\nfailure modes:\n\n| Case | What it probes | Result |\n|---|---|---|\n| A1–A3 | Near-identical twins: `fetch_url` vs `read_file` vs `read_file_lines` | 3/3 |\n| B1, B2 | Pure knowledge question — must call nothing | 2/2 |\n| C1 | Missing information — must decline rather than invent | 1/1 |\n| D1, D2 | Arguments buried in messy prose; append vs overwrite | 2/2 |\n| F1, F2 | Blocking vs background, disambiguated by \"it takes six hours\" | 2/2 |\n\n**12/12.** This refutes a claim made earlier in this same investigation — that pruning a\nharness's toolsets would reduce tool-selection errors. It does not. Pruning is still worth doing, but\n*only* to reclaim context.\n\n| Harness | Easy task | Hard task | Fixed context | Algorithm chosen |\n|---|---|---|---|---|\n| opencode 1.18.18 | 10/10 · 27 s | 14/14 · 213 s | 7,447 | heapq + in-degree, O(n log n) |\n| Atomic Agent v0.2.1 | 10/10 · 46 s | 14/14 · 157 s | 5,399 | dict of pending nodes |\n| Hermes v0.20.2 | 10/10 · 60 s | 14/14 · 152 s | 14,032 | naive rescan, O(n²) |\n\nAll three passed everything, including the hard task's reversal semantics. Neither task discriminated on\ncorrectness — the outputs are genuinely independent (10–24% textual similarity, i.e. shared boilerplate).\n**Fixed context cost is what separates them**: 4.1%, 5.7% and 10.7% of a 131K window burned\nbefore any work begins, and the Hermes figure is already after pruning six toolsets, which recovered 7,973\ntokens per session.\n\n**Two things had to be fixed before opencode could be measured at all.** It shipped\nwith no configuration pointing at a local model — its `auth.json`\n\nlisted only a cloud provider —\nand its database did not migrate across the 1.2.18 → 1.18.18 upgrade, failing every run with\n`no such column: name`\n\n.\n\nAcross every set, the failures cluster into two kinds.\n\n| Failure | correct | given | Diagnosis |\n|---|---|---|---|\n| 200-term modular recurrence, budget 32,768 | 98,286 | 60,092 | exact arithmetic depth — 16,819 thinking tokens, `finish_reason: stop` , still wrong |\n| Knapsack, 1 of 3 attempts | 30 | 35 | constraint violated — picked 5+4+3 = weight 12 against a capacity of 10 |\n| 250-step simulation, effort low | 1,932,916 | 1,039,216 | insufficient thinking for the depth |\n\nThe pattern: it fails when it must **hold a constraint while optimising**, and when exact\narithmetic runs very deep. More thinking budget does not fix either — the recurrence failed with 16,819\nthinking tokens and a clean self-termination.\n\n**When it fails, it fails loudly.** Of ten wrong answers collected across the session, nine were\noff by 25% or more — typically 39% to 99%. Only one was a near miss. That is a practically useful property: a\nplausible-looking number is probably right, and a wrong one usually looks absurd. *Scope: numeric problems.\nFailure modes in prose or code may differ entirely.*\n\nA script generated five fresh problems with a clock seed, writing the questions to one file and the answers\nto another that was not read. **Opus 5 max** answered them by reasoning alone, with no code\nexecution, and committed its answers to a hashed file *before* any verification existed. The local model\nthen received the same five with an explicit instruction to solve them mentally.\n\n| P | correct | Opus 5 max | Qwen3.8-27B local |\n|---|---|---|---|\n| 1 | 9955 | 9955 ✓ | 9955 ✓ |\n| 2 | 343 | 343 ✓ | 343 ✓ |\n| 3 | 22 | 22 ✓ | 22 ✓ |\n| 4 | 26 | 26 ✓ | 26 ✓ |\n| 5 | 1065 | 1065 ✓ | 1065 ✓ |\n\n**Read this correctly.** Five for five on both sides means **neither model\nfailed on this set** — a statement about the set's difficulty, not about model equivalence. Three\nasymmetries remain even with the commit-then-verify protocol: the problem *types* were chosen by Opus 5\nmax, the sample is five, and the hardware is not comparable. A tie proves nothing except that these five\nproblems do not separate them.\n\n**And a broader benchmark does separate them.** Artificial Analysis's Intelligence\nIndex v4.1.1, which aggregates nine evaluations, places **Qwen3.8-27B at 52** against\n**Claude Opus 5 (max) at 63** — level with GPT-5.6 Luna, eleven points below the top. That is a\nthird-party figure this benchmark did not verify, but it is the right corrective: a five-problem tie measures\nthe problems, not the models. What the local model demonstrably is, on this hardware, is strong enough that\nfinding its ceiling took three escalations of difficulty. What it is not is frontier-equivalent.\n\nA 27B model with a **131,072-token window and vision** at 64 tok/s sustained on a consumer card\nis not what a 27B usually buys. The reason is architectural: only about one layer in four keeps a KV cache, so\ncontext costs 28,672 B/token instead of the ~90 KB a dense 27B would need. Long context here is a property of\nthe architecture, not of tuning.\n\nCapability was rarely the limit. It solved every coding task, every tool-calling trap, and matched a frontier model on a five-problem set. It broke only on very deep exact arithmetic and on holding a constraint while optimising.\n\nEvery meaningful decision was a VRAM trade, not a throughput trade. q8_0 KV would cut the window from 131K to ~52K. The vision projector costs 1,125 MiB. The 765 MiB between two quantisations of the same model is worth 28,000 tokens. Meanwhile the desktop itself moves 500 MiB between an idle session and a working one. Speed was never the problem: a complete working module in 27 seconds is not a bottleneck.\n\n`GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32`\n\n`--spec-type draft-mtp --spec-draft-n-max 2`\n\n`-np 1`\n\n`xhigh`\n\ndestroyed 6 of 9 attempts at one task.`-c`\n\nhigh| Claim | What measuring showed |\n|---|---|\n| \"n=3 is the MTP sweet spot\" | True on an RTX 6000. On a 3090, n=2 — the optimum tracks spare verification compute |\n| \"Use q8_0 KV\" | Doubles KV cost; drops the window from ~131K to ~52K |\n| \"6K–12K is the optimal reasoning budget\" | No fixed optimum exists. The requirement scales 312 → 21,330 tokens with problem size |\n\"Lower `-c` to gain speed\" | Allocation costs 1.2% between 8K and 131K. The gains come from filling less |\n| \"MTP costs 2–3 GB of VRAM\" | 834 MiB measured |\n\"`--spec-draft-p-min` gives 15%\" | 2.4–3.3%, inside or barely outside noise |\n| \"Temperature is the quality knob\" | It is a speed knob: it moves draft acceptance, not answer correctness |\n| \"Fewer tools means fewer tool-calling errors\" | 12/12 on adversarial cases with 12 tools; perfect with 30 easy ones |\n| \"Prefer Maximum Performance in PowerMizer\" | Under load the GPU is already at its top level; it only raises idle heat |\n\nThe 1M-token windows advertised for this class of model are not a configuration away. At the measured\n28,672 B/token, one million tokens needs **26.7 GiB of KV alone** — more than the whole card,\nbefore weights. Even the native 262,144 does not fit (7.0 GiB of KV against 6.1 GiB available). Pushing KV to\nsystem RAM with `-nkvo`\n\ndoes not rescue it: 26.7 GiB against 31 GiB of RAM would not fit alongside\nthe OS, and if it did, reading that KV per generated token over PCIe 4.0 x16 lands at\n**0.75–0.94 tok/s**. Those windows run on 80 GB cards. On a 3090 it is arithmetic, not tuning.\n\nAlso out of reach: the FP8 and NVFP4 builds of this exact model, which are published and require Ada or Blackwell; and undervolting, which needs an X server.\n\nSingle machine, and most configurations measured once rather than repeated — the\nthermally-equalised comparisons and the multi-attempt coding runs are the exceptions. Perplexity is a proxy for\nquantisation damage, not a functional quality benchmark. The harness comparison rests on context cost and one\nobservation about algorithm choice, because neither coding task discriminated on correctness. Reasoning and\ntemperature findings cover **verifiable single-answer problems and code generation only** — for\nopen-ended work such as prose or visual design, where many outputs are valid, this benchmark says nothing, and\ntemperature very plausibly behaves differently there.\n\nKept deliberately. Eight of these produced a stated conclusion that later measurement overturned.\n\n`max_tokens`\n\n. It terminates at ~25,772 thinking tokens.\n\n```\nexport GGML_CUDA_CUBLAS_COMPUTE_TYPE=fp32   # or every image request aborts\n\nllama serve -m Qwen3.8-27B-AD-Q4_K_M.gguf \\\n  --mmproj mmproj-F16.gguf \\\n  -ngl 999 -fa on --jinja -np 1 \\\n  -c 131072 -ub 512 -b 4096 \\\n  --cache-type-k q4_0 --cache-type-v q4_0 \\\n  --spec-type draft-mtp --spec-draft-n-max 2 \\\n  --reasoning off \\\n  --temp 0.7 --top-p 0.95 --top-k 20 --min-p 0.0 \\\n  --host 0.0.0.0 --port 8080\n```\n\n`-np 1`\n\n— more slots disable MTP.`q4_0`\n\nKV — q8_0 costs 57,344 B/token and drops the window to about 52K.`-ub 512`\n\n— 1024 gives the same prefill and costs 404 MiB.`--reasoning off`\n\n— for coding. Switch on with an unbounded budget only for hard logic.`chat_template_kwargs`\n\n, not the standard field.The full suite — problems, reference implementations, hidden test suites, probe scripts and raw logs — is designed to be run against any other model and to be broken. A benchmark nobody can falsify is worth nothing.\n\n`llama perplexity`\n\non identical corpora and settings. Coding tasks scored by hidden suites validated\nagainst reference implementations, two of them additionally against exhaustive brute force. Reasoning answers\nverified by executable ground truth. GPU telemetry sampled by a single long-lived process at 0.0% measured CPU\noverhead. Where a run lacks per-item telemetry, section 02 says so.", "url": "https://wpnews.pro/news/qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths", "canonical_source": "https://jonidimo.github.io/qwen38-3090-benchmark/benchmark.html", "published_at": "2026-08-17 23:11:52+00:00", "updated_at": "2026-08-17 23:41:16.313649+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research"], "entities": ["Qwen3.8-27B", "NVIDIA GeForce RTX 3090", "llama.cpp", "unsloth", "AtomicChat", "Debian GNU/Linux 13", "AMD Ryzen 9 5950X", "CUDA"], "alternates": {"html": "https://wpnews.pro/news/qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths", "markdown": "https://wpnews.pro/news/qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths.md", "text": "https://wpnews.pro/news/qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths.txt", "jsonld": "https://wpnews.pro/news/qwen3-8-27b-on-a-single-rtx-3090-crash-fix-131k-context-9-myths.jsonld"}}