{"slug": "beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy", "title": "Beating benchmarks by weight-space optimizing decision models with DSPy", "summary": "Compound AI released dspy-wsp, a prototype DSPy optimizer whose WeightSpace component uses a writer LLM (GPT-6-Astra) to edit a smaller model's weights directly, with no gradients and no training. Tested on five open decision models from 0.6B to 9B parameters against the Typed Decisions benchmark, WeightSpace alone added 1.9 to 16.1 points of accuracy on 400 unseen test cases, and combined with the ReAnchor optimizer it added 6.5 to 20.0 points, at roughly $5–10 per model for 25 forward passes. The release builds on weight-space programming research by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab, whose earlier experiment had GPT-6-Astra edit 0.016% of Qwen3.5-0.8B's weights to raise its MATH-500 score from 7.9% to 48.1%.", "body_md": "Last week, Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab showed that frontier models can edit the weights of smaller models directly, with no gradients and no training, using a technique called “[Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/).” In one experiment, GPT-6-Astra edited 0.016% of Qwen3.5-0.8B’s weights, eliminating a looping problem and raising its MATH-500 score from 7.9% to 48.1%. Haoze “Hector” He [applied the weight-space technique to decision models](https://x.com/HaozeHe_Hector/status/2118048048148623453), demonstrating it works there, too.\n\nIt’s early days, but models that can perform brain surgery on one another have mind-boggling potential.\n\nToday we’re releasing [`dspy-wsp`](https://github.com/cmpnd-ai/dspy-wsp), a prototype DSPy optimizer that brings weight-space programming to your existing programs. The `WeightSpace` optimizer can run Rikiya’s LLM example, but it does its best work with decision models.\n\nWe tested `WeightSpace` on five open decision models, from 0.6B to 9B parameters, with the Typed Decisions benchmark. For each model, GPT-6-Astra worked through 25 forward passes at maximum reasoning effort for about $5–10. We scored each edited model on 400 test cases it never saw during optimization.\n\nEvery model improved: `WeightSpace` alone added between 1.9 and 16.1 points of accuracy, and when combined with [ReAnchor](building-jev-programs-with-dspy.html), a DSPy optimizer that refits each question’s decision thresholds, it added between 6.5 and 20.0 points. (View all our scores over at [the repo](https://github.com/cmpnd-ai/dspy-wsp).)\n\n## How WeightSpace works\n\nEach `WeightSpace` run hands your model’s weights to a writer LLM. The writer works in turns. Every turn opens with a short explanation of what the writer will try, followed by one block of code: a Python block that reads or edits weights in a sandboxed kernel, or a forward block that runs your program on practice examples. Forward passes come from a fixed budget, and when the budget runs out, the writer’s best edit becomes your patch.\n\n## Using `dspy-wsp`\n\n`dspy-wsp` works with [DSPy 3.5](dspy-3-5.html). Install it with pip:\n\n```\npip install dspy-wsp\n```\n\nWeightSpace takes any DSPy module and any metric. You call compile as you would with any other DSPy optimizer, and you get back a module that runs on edited weights.\n\n``` python\nimport dspy\nfrom dspy_wsp import WeightSpace, load_decision_backend\n\nbackend = load_decision_backend(\"d1\")   # d1-omni-600M\noptimizer = WeightSpace(\n    metric=metric,\n    writer_lm=dspy.LM(\n        \"openai/gpt-6-astra\",\n        model_type=\"responses\",\n        reasoning_effort=\"max\",\n    ),\n    backend=backend,\n    max_passes=25,\n    holdout=0.25,\n    run_dir=\"runs/triage\",\n)\ncompiled = optimizer.compile(\n    dspy.Predict(Triage), trainset=train, valset=test\n)\n\n# Save the edit\ncompiled.wsp_patch.write_checkpoint(\n    backend.weights_dir, \"edited-model/\"\n)\n```\n\nThe edited model runs on your own hardware. WeightSpace loads the student’s weights into the Python process, and the compiled module calls that local model in place of a hosted API. On a Mac with Apple silicon, the model runs on the built-in GPU through MLX. On Linux, it runs through PyTorch, on an NVIDIA GPU if you have one. These models are small: we ran every experiment in this post on one Mac, and the 2B models used about 6 GB of memory.\n\nFor text-generating LLMs, which need many long answers per pass, WeightSpace can also drive vLLM on NVIDIA GPUs. When you finish, you can save the edit as a small patch or write out a full checkpoint, which serves like any other model.\n\nHere’s how you load the edited model we saved above:\n\n``` python\nimport dspy\nfrom dspy_wsp import load_decision_backend\n\nfrom triage import Triage   # your signature\n\nbackend = load_decision_backend(\"edited-model/\")\n\nprogram = dspy.Predict(Triage)\nprogram.set_lm(backend.lm())\n\nresult = program(\n    ticket=\"My invoice shows a charge I don't recognize.\"\n)\n```\n\n## Try it and tell us how it goes\n\n`dspy-wsp` is very experimental. It runs on several inference backends, and we’ve tested it in only a handful of environments and model families. Editing runs on text-generating LLMs are long and costly, and they may not produce compelling results. We wanted to get it out anyway, so you can experiment with it, modify it and play with it. Tell us where it works and where it doesn’t, and send issues and pull requests to [the repository](https://github.com/cmpnd-ai/dspy-wsp).\n\nEven so, we’ve found `dspy-wsp` surprisingly practical for improving decision-model programs, and we’ve been impressed by how well it composes with ReAnchor.\n\nIf you have a Mac or a GPU with at least 4 GB of memory, you can run weight-space optimizations on d1-omni-600M.\n\nThanks to [Rikiya Takehi](https://rikiyat.github.io), [Jayden Teoh](https://jaydenteoh.github.io/), [Alex Zhang](https://alexzhang13.github.io/) and [Omar Khattab](https://omarkhattab.com/), whose research this builds on; to [Haoze “Hector” He](https://x.com/HaozeHe_Hector), for his earlier work editing decision models; and to [Michael Isaac](https://x.com/michaelnisaac), who contributed the GEPA search option.", "url": "https://wpnews.pro/news/beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy", "canonical_source": "https://www.cmpnd.ai/blog/weight-space-programming-dspy.html", "published_at": "2026-11-13 17:43:26+00:00", "updated_at": "2026-11-13 17:49:57.530962+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-tools", "ai-research", "developer-tools"], "entities": ["Compound AI", "DSPy", "dspy-wsp", "WeightSpace", "GPT-6-Astra", "Qwen3.5-0.8B", "Rikiya Takehi", "Omar Khattab"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy", "markdown": "https://wpnews.pro/news/beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy.md", "text": "https://wpnews.pro/news/beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy.txt", "jsonld": "https://wpnews.pro/news/beating-benchmarks-by-weight-space-optimizing-decision-models-with-dspy.jsonld"}}