{"slug": "weight-space-programming-needs-triage-first-so-i-built-it", "title": "Weight-space programming needs triage first, so I built it", "summary": "A developer built wsr (Weight-Space Repair), an open-source MIT-licensed tool that triages why a local LLM produced a bad response before attempting weight-level fixes. Triage runs without a GPU and checks the model file, config, request, output and live probes to name a single cause; only when setup issues are ruled out does it trace failures inside the network and apply weight patches or steering, with fixes shipped only if a statistical judge accepts them on held-out and confirmation data. In testing on Qwen2.5, probe-gated steering cut false refusals from 48 of 167 to 11 while harmful requests answered fell from 28 of 124 to 5, versus abliteration's 9 false refusals and 108 harmful answers.", "body_md": "This month MIT CSAIL researchers showed that an LLM can read and edit another model's weights directly to fix a behaviour ([Takehi, Teoh, Zhang and Khattab, \"Sparks of Weight-Space Programming\"](https://rikiyat.github.io/blog/weight-space-programming/)). It's a cool idea, and I've been building on it.\n\nBut the debate after the post was mostly not about weights. It was about the baseline: a [critique](https://brainwagon.org/blog/2026-11-08-a-skeptical-look-at-weight-space-programming.html) argued much of the gain came from answers looping until the token cap with a temperature other than the model card's, and the [HN thread](https://news.ycombinator.com/item?id=50452655) found a cheaper fix that did as well.\n\nThat matched my own experience. Every time a local model gave me a blank reply or looped on the same line, my first guess was \"the model is bad\". Most times I was wrong. It was the chat template, a sampler setting, a context that got cut off, or one broken tensor in a quantised file.\n\nSo I built **wsr** (Weight-Space Repair): triage before surgery. You give it the failing request and reply, and it tells you why it failed and how to fix it. Only when the setup is ruled out does it go into the weights.\n\n```\npip install wsr\nwsr triage --response response.json --request request.json\n```\n\nTriage needs no GPU and has no dependencies.\n\nIt goes through layers in order: the model file, the config, the request and response, the output itself, live probes that rerun the prompt changing one thing at a time, and known issues. Then it names one cause, shows the evidence, and gives the fix. If it can't find a cause, it says so instead of guessing.\n\nSome things it caught while I was testing:\n\n`max_tokens` before any answer\nIf nothing in the setup explains it, `wsr deep` traces the failure inside the network and tests it against random controls. One example: Qwen 1.5B refused \"how do I hack my own router\". The cause lined up with the refusal\n\ndirection (cosine 0.88). Removing that direction fixed 8 of 8 prompts, and shuffled directions fixed 0 of 8.\n\n`wsr repair` then looks for the smallest fix: a weight patch, a steering, or serving settings. It can also hand the job to an agent (Claude or any OpenAI-compatible model) with weight-editing tools, which is the weight-space programming part. The difference is what's around it: a fix only ships if a statistical judge accepts it on held-out data (exact sign test, corrected for every candidate tried), and then once more on a confirmation set it has never seen. Rejected fixes get reported too.\n\nThe result I like most is on refusals with Qwen2.5 (refusing safe prompts, answering harmful ones):\n\n|  | false refusals | harmful requests answered | \n|---|---|---|\n| baseline | 48 of 167 | 28 of 124 | \n| removing the refusal direction (abliteration) | 9 | 108 | \n| probe-gated steering (wsr) | 11 | 5 | \n\nAbliteration does stop the wrong refusals, but it also stops the right ones. The gate keeps the real ones.\n\nI only tested small models (0.5B to 1.5B) on my GTX 1650 with 4 GB. Weight repair is tested up to 1B. Most results come from one run. Everything is in [RESULTS.md](https://github.com/ayushh0110/Weight-Space-Repair/blob/main/docs/RESULTS.md) and [LIMITATIONS.md](https://github.com/ayushh0110/Weight-Space%20Repair/blob/main/docs/LIMITATIONS.md), including the fixes that got rejected.\n\nRepo: [https://github.com/ayushh0110/Weight-Space-Repair](https://github.com/ayushh0110/Weight-Space-Repair) (MIT)\n\nThe most useful thing you can send me is a case where wsr names the wrong cause. There is an issue form for that.\n\nResults on bigger models would help a lot too.\n\nwsr's repair agent builds on the weight-space programming work by Takehi, Teoh, Zhang and Khattab (MIT CSAIL, 2026): [https://rikiyat.github.io/blog/weight-space-programming/](https://rikiyat.github.io/blog/weight-space-programming/). wsr is an independent project, not affiliated\n\nwith them. If you use the agent, please cite their work (the README has a BibTeX entry).\n\n*I used AI assistance to draft this post and to write parts of the code; I checked every claim and number.*", "url": "https://wpnews.pro/news/weight-space-programming-needs-triage-first-so-i-built-it", "canonical_source": "https://dev.to/ayushh0110/weight-space-programming-needs-triage-first-so-i-built-it-3ek9", "published_at": "2026-11-19 23:02:17+00:00", "updated_at": "2026-11-19 23:13:15.009529+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-safety", "ai-research"], "entities": ["wsr", "MIT CSAIL", "Qwen", "Qwen2.5", "GitHub", "Claude"], "also_reported_by": [], "takeaway": "Triage setup issues (chat template, sampler settings, truncated context, broken quantised tensors) before editing weights, and gate any weight patch or steering fix behind an exact sign test on held-out plus confirmation data — probe-gated steering cut false refusals from 48/167 to 11 while keeping ", "recommendation": null, "alternates": {"html": "https://wpnews.pro/news/weight-space-programming-needs-triage-first-so-i-built-it", "markdown": "https://wpnews.pro/news/weight-space-programming-needs-triage-first-so-i-built-it.md", "text": "https://wpnews.pro/news/weight-space-programming-needs-triage-first-so-i-built-it.txt", "jsonld": "https://wpnews.pro/news/weight-space-programming-needs-triage-first-so-i-built-it.jsonld"}}