{"slug": "how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb", "title": "How to run OxCoder-9B Q5_K_M on 2× RTX 5060 Ti 16 GB", "summary": "A three-run evaluation of OxCoder-9B Q5_K_M on a 2× RTX 5060 Ti 16 GB tensor-split pair (32 GB total) delivered a 0/3 reply pass rate, 0/4 units and 0/4 tests, with every run capped at 261,000 tokens against the served 262,144-token slot. The needle-in-haystack probe hit 28 of 30 anchors, 3/3 at every depth through 90% of the window, while prefill fell from 4,229 to 2,364 tok/s (−44%) and decode from 99.2 to 53.0 tok/s (−47%) between the 10% rung and the full slot. The author attributes the failures to the eval-to-implementation handoff rather than the eval, citing run 1's self-revision that deletes twelve @spec lines and defines an eviction Task whose loop is undefined at kv_1b.ex:19, and recommends routing no implementation work through the model, using it instead as a long-document reader.", "body_md": "Three v5 runs on the 32 GB tensor-split pair, each capped at 261,000 tokens, i.e. the served 262,144 slot minus a margin. The cap tracks the window on purpose: a short mid-answer cap can kill a run the model would have finished, and a run is only comparable to the slot it was given.\n\nThe pipeline never delivered. Run 2 is the spiral: ~151 k reasoning tokens against a 1,800 s wall, zero content — the slot postmortem showed 152,208 tokens occupied before the abort freed it. Run 1 emitted module and test, then re-emitted both inside the same reply — four fences where the contract says two — and the revision is worse: all twelve `@spec` lines deleted, eviction “fixed” as a `Task` whose recursive lambda references an undefined variable and whose `self()` is the task, not the server. Run 3 delivered its review and both modules as flat prose with zero fences — two code pairs whose only difference is two comment lines — with a `\n\n` token stranded mid-content, the answer channel leaking its own scratchpad.\n\nThe needle is where it earns its keep. One anchor, buried at every tenth of the served slot, three repeats per depth, each request opening with a fresh nonce so every step measures cold prefill. 28 of 30 hit, 3/3 at every depth through 90 % of the window; the 100 % edge is a coin flip — the hit found the marker, the two misses burned the 512-token cap re-analyzing the request (“The user wants to find a specific marker hidden in the text…”) and truncated before transcription. Prefill decays 4,229 → 2,364 tok/s (−44 %) and decode 99.2 → 53.0 tok/s (−47 %) from the 10 % rung to the full slot: smooth, no cliff, no cache hole. The short-probe figure of 165 tok/s at n=105 tokens ages badly: at 26 k real decode is 99 tok/s, at the full slot 53. The ladder is the honest one.\n\nMethodology, disclosed: the served recipe runs `--reasoning on` with a 512-token answer cap, and 26 of the 28 hits were recovered from the reasoning stream before truncation (28 of the 30 rows finished `length`). Grounding is N/A: completion-only harness, no tools wired.\n\n## The catch\n\nDelivered-pipeline pass rate: 0/3 replies, 0/4 units, 0/4 tests.\n\n- run 1 pair A (`kv_1a.ex` ) — twelve`@spec` s for`put/3,4` ,`get/2` ,`delete/2` ,`stats/1` and zero`def` s behind them;`init/1` returns a 5 s timeout with no handler, so every instance crashes after 5 s idle; the promised periodic eviction is unreachable code.\n- run 1 pair B (`kv_1b.ex` ) — the self-revision that deletes the typespecs: eviction as a`Task` whose`loop` is undefined inside its own definition (`:19` ), and whose`self()` points at the task, not the server. Unreachable code became uncompilable code.\n- run 3 pairs A/B (`kv_3a.ex` ,`kv_3b.ex` ) —`:timer.now() + current_ttl` integer arithmetic on a now-triple (`:97,110` ), an eviction timer that fires a shape nothing matches (`:61,74` ),`stats()` hard-coded to`__MODULE__` ; pair B is pair A with two comment lines changed.\n- Tests — `test_1a.exs` reuses the first block’s`pid` binding in every later body;`test_1b.exs` calls a 4-tuple against a 5-tuple head;`test_3a.exs` and`test_3b.exs` are SyntaxErrors.\n\nThe one honest positive: run 1’s thirteen-line review enumerates every trap with correct arities and semantics — a design doc its own code then ignores. The eval→implement handoff is the broken organ, not the eval.\n\n## Representative output\n\nRun 1, pair B — the self-revised eviction, trimmed to the Task:\n\n``` python\ndef init(opts) do\n  default_ttl = Keyword.get(opts, :default_ttl, @default_ttl)\n  # Start eviction task\n  Task.start_link(fn ->\n    loop = fn ->\n      send(self(), :evict)\n      :timer.sleep(60_000)\n      loop.()\n    end\n    loop.()\n  end)\n```\n\n`loop` is referenced inside its own definition before it is bound — a compile error at `kv_1b.ex:19`. Even if it compiled, `self()` inside the task is the task’s pid, so `:evict` would never reach the GenServer.\n\n## Route\n\nRoute no implementation work through it. The fit is as a long-document reader: the full 262,144-token window fits the pair, 9 of 10 depths retrieve cleanly, and decode still runs 53 tok/s at the slot edge. It reads long documents well and cannot write Elixir you can merge; keep a compile gate between it and any repo.", "url": "https://wpnews.pro/news/how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb", "canonical_source": "https://overbring.com/ai-benchmarks/2026-11-12-prithivmlmods-oxcoder-9b-q5_k_m-gguf-rtx-5060-ti-16gb-x2-tensor-split-llama-server-cuda/", "published_at": "2026-11-11 23:47:59+00:00", "updated_at": "2026-11-12 18:23:18.247610+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-infrastructure"], "entities": ["OxCoder-9B", "RTX 5060 Ti", "kv_1a.ex", "kv_1b.ex", "kv_3a.ex", "kv_3b.ex"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb", "markdown": "https://wpnews.pro/news/how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb.md", "text": "https://wpnews.pro/news/how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb.txt", "jsonld": "https://wpnews.pro/news/how-to-run-oxcoder-9b-q5-k-m-on-2x-rtx-5060-ti-16-gb.jsonld"}}