# Teaching a local coding agent from its own mistakes: DPO on a 30B model > Source: > Published: 2026-08-31 11:38:04+00:00 The **WUIC Assistant** is our agentic VS Code plugin: it scaffolds Angular components, dashboards, reports, workflows, and metadata patches for apps built on the WUIC framework. Under the hood it runs **qwen3-coder:30b** — a Mixture-of-Experts model with ~3B active parameters — [served locally through Ollama](https://wuic-framework.com/blog/local-llm-ollama-mcp-agentic-vscode) on a single RTX 4090. No API calls, no data leaving the machine. For weeks the plugin had been sitting at **100% on its curriculum**: 32 tasks, five green runs each, a 96/96 regression suite. Beautiful numbers, and a problem. A curriculum that always passes has stopped measuring anything. It could no longer tell us whether the model *generalized* to the prompts a real developer types, nor how often it took a wrong turn before correcting itself. We wanted two things: `STOP: use scaffold, not ng generate` ), and only This post is the story of how we got there. It involves five OutOfMemory crashes, a saturated benchmark, two hidden contaminations, and a 30-billion-parameter model that did learn — though not exactly what we asked it to. A model that evaluates itself on the data it was aligned to is lying to itself. So we built **eval-only** sets, with one iron rule: *never in training, never curricularized*. If you stabilize a task by adding a dedicated mechanism, that task loses its value as a measurement — and it gets replaced. The latest of these, the fourth **"HARD"** set (Q01–Q12), deliberately raises the bar on the levers where the base model actually struggles: Measured on a clean bench, the base model lands at **25/36 (69%)** — failing Q02, Q04 and Q07 on every attempt, and Q05 on two of three. But the most interesting data point was elsewhere: **Q01 passed 3 times out of 3 — with 9 redirects**. It got there, but by bouncing off the guards the whole way. That's the churn we wanted gone. Before touching the model, a deceptively innocent question — *"is the test environment actually clean and in sync?"* — opened a can of worms. **Contamination #1: the metadata reset.** Our between-runs reset ran `DELETE WHERE id > baseline` — it deleted the rows that had been *added*, but never reverted the **UPDATEs** (like `hide_in_list` ) applied to pre-existing rows. Run after run, columns on the `customers` table were progressively hidden, until `route_columns` started answering "no columns found." Worse: some of our test prompts had been reworded over time to *compensate* for this degradation — meaning we'd been fixing the symptoms of an environment bug and mistaking it for backend behavior. The fix: restore the full column set from the clean baseline (`metadataTutorial` ) on every reset. The general lesson we kept: **a reset has to cover what you mutate, not just what you add.** **Contamination #2: the report generator.** A shared file (`gen-report-from-template.mjs` ) had been overwritten by a model run, shrinking from 246 lines to 60 — and our reset didn't cover that directory. Restored, added to the reset scope, and write-guarded. The moral: **before you trust the numbers, trust the environment.** A test bench that drifts slowly produces phantom regressions that look exactly like model bugs. With a clean bench, the third holdout set went from 22/36 to **36/36**. The fixes were **generic** — synonyms in the detectors, correct precedence between task "kinds", scaffold guards for reports — never a patch for an individual prompt. That distinction is the whole game: you generalize the *root condition*, you don't enumerate the cases. But the churn remained. And churn isn't solved by adding one more guard — it's solved by teaching the model not to fumble the first move. That meant touching the model itself. The right technique here isn't more supervised fine-tuning. The SFT we'd already tried **masked** the rejected turns — the model never got the signal "*this* move is wrong." Redirects dropped only from 43 to 39. **DPO** (Direct Preference Optimization) learns from the **pair** instead: for the exact same context, *prefer* the accepted action over the rejected one. It's a perfect fit for our problem, because every redirect in the historical transcripts is already a ready-made pair: `ng generate component…` → STOP) = `scaffold widget=map` → Saved) = An extractor pulled **559 pairs** from more than a thousand archived runs (including the red ones — a pair is valuable if it contains a valid rejected→corrected transition), deduplicated and balanced so no single task dominates. Here's where theory meets 80 GB of VRAM. DPO does **four forward passes** per step (policy and reference, over chosen and rejected). And our system prompt — the `.clinerules` that give the model all the WUIC context — is **enormous**: ~20,000 tokens. Multiplied by four passes, the 30B model went OOM *before step 0*, five times in a row, first on a 4090 under WSL2 and then even on an **80 GB A100**. The debug path, condensed: `peft` .`[seq × vocab≈150k]` → OOM.`lm_head` + loss without ever materializing the full logits) promised to solve it… but on a 4-bit quantized `PeftModel` it was `SMOKE_FAIL` with no clear traceback.The breakthrough was a counterintuitive one: **the real problem wasn't DPO, it was the system prompt.** We **trimmed the system block in the dataset** (head + tail, the parts that matter), bringing sequences from ~24k down to **3–5k tokens**. At that point the fp32 logits take ~6 GB — they fit comfortably in 80 GB — and the liger fused loss was no longer needed. In fact it was the very cause of the last failure: **disabled**, standard DPO ran clean. The final smoke test: 8 steps, loss trending down, positive `rewards/margins` , `accuracies` 0.75–0.88. The model preferred the *chosen*. Green light. The full run trained in the background on the A100 (detached, to survive SSH drops), monitored step by step: | Step | Loss | rewards/accuracies | rewards/margins | |---|---|---|---| | 5 | 0.62 | 0.63 | +0.30 | | 20 | — | 0.73 | +0.59 | | 46 | 0.47 | 0.78 | +0.74 | | 56 | 0.41 | 0.88 | +1.02 | **70 steps, 1 epoch over 559 pairs, 4h11m, zero OutOfMemory** from start to finish — even when VRAM read 96% (a false alarm: that was PyTorch's stable *reserved pool*, not a live allocation at the edge; the paged 8-bit optimizer spills to system RAM as a relief valve). Margins above 1.0 and accuracies at 88% say the preference signal was learned cleanly. The adapter was merged, converted to GGUF q4_K_M, and served by Ollama as `qwen3-coder-wuic:30b-dpo` . Then came the **eval gate**, with criteria fixed *before* looking at the results: A/B on the same clean bench, three rounds per task: | Criterion | Base | DPO | Gate | Verdict | |---|---|---|---|---| | PASS on HARD | 25/36 (69%) | 28/36 (78%) | ≥ 25/36 | pass | | Curriculum | 96/96 | 96/96 | 96/96 | pass | | Report guard | intact | intact | intact | pass | | Redirects/run | 0.67 | 0.83 | < 0.47 | fail | | Zero-redirect runs | 53% | 39% | > 53% | fail | The capability gains are real: Q02 went from 0/3 to **3/3** — a task the base never solved — and Q05 from 1/3 to 2/3, with zero forgetting. And on Q01, the flagship churn case the pairs were built around, redirects fell from 9 to 6. But the primary metric — the whole reason we ran DPO — moved the *wrong* way: +25% redirects overall. Digging in, the number is less damning than it looks. The base scores zero redirects on Q02 because it *fails cleanly*; the DPO model racks up four because it *fights and wins* (0/3 → 3/3). A redirect count that punishes engaging with hard tasks is a metric with a blind spot — a flaw we only saw because the gate forced us to stare at it. The pre-registered rule, though, was: no automatic adoption unless the gate clears. It didn't clear. The base stays the plugin's default; the DPO model sits one setting away, and switching in either direction is trivial — exactly the cheap, reversible outcome the gate was designed to buy. One epilogue. On a different job — single-shot tool routing for the [in-product RAG chatbot](https://wuic-framework.com/blog/local-llm-ollama-mcp-agentic-vscode), where there are no guards and no multi-turn churn to fall back on — the same DPO model scored **96/102 against the base's 88/102** at identical speed, and *that* gate it cleared. It now ships as the chatbot's default brain. The model didn't fail; it turned out to be better at a different job than the one we trained it for. Because that's the real point of this whole story. Not "we fine-tuned a model," but: **we built a ruler that doesn't lie, we cleaned the test bench, we fixed the criteria before looking — and we let the numbers decide.** Twice. The model is public. `qwen3-coder-wuic:30b-dpo` is on [Hugging Face](https://huggingface.co/castricolorenzo/qwen3-coder-wuic-30b-dpo) as a GGUF (q4_K_M, ~19 GB) — pull it straight into Ollama and run it on your own 24 GB GPU. The full setup, end to end — the Ollama tuning, the WUIC backend config, and the WUIC Assistant VS Code extension — is on the [Run the WUIC coding model locally](https://wuic-framework.com/model) page. No API calls, no data leaving your machine.