{"slug": "does-weight-space-programming-need-to-touch-the-hidden-layers", "title": "Does weight-space programming need to touch the hidden layers?", "summary": "Editing only three output-layer rows of Qwen3.5-0.8B — the rows scoring the `</think>` stop token and two spellings of `Wait` — reached 55.2% (276/500) on MATH-500, compared with 52.2% (261/500) for the published Astra edit that modifies feed-forward layers 15 and 19, according to a 17 November 2026 blog experiment. The author cautions the runs used different engines, with an exploratory paired bootstrap interval of −0.4 to +6.4 points and a descriptive McNemar p of 0.110, so the result does not establish superiority or equivalence. The output-head edit averaged 15,340 tokens versus about 21,638 retokenized reference tokens on the 20 questions with published reference text, 29.1% fewer on that subset, while the unedited model scored 7.0% (35/500).", "body_md": "An output-head experiment · 17 November 2026\n\n# Does weight-space programming need to touch the hidden layers?\n\nI tried leaving Qwen’s internal layers alone and editing only three rows that score its next token. The model reached 55.2% on MATH-500. Then I looked at the runs that still would not stop.\n\nIn [Sparks of Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/), Astra improved Qwen3.5-0.8B by editing two internal feed-forward layers. [Nathan’s follow-up](https://nathan.rs/posts/weight-space-lp/) showed how a linear program could find a related edit. My experiment asks a narrower question: how much of the result can we recover at the model’s final output layer?\n\nA reasoning model already has a token for ending its scratchpad: `</think>`. I fitted small linear classifiers to the model’s final hidden state, then folded their weights into the rows for that token and two spellings of `Wait`. This changes when it stops or says “Wait” while preserving every original transformer tensor. The edited weights run without a separate classifier at inference time.\n\nOn the full 500-question benchmark, this candidate scored **55.2%**, alongside **52.2%** for the selected published Astra-edited reference run. The runs use different engines, so this is evidence worth investigating—not proof of a better method. The more surprising result was where the remaining loops went.\n\n## Three output rows. 500 math questions.\n\nAll rows below cover the same 500 MATH-500 questions. Each uses one selected answer per question. The references come from external runs; this is a benchmark comparison, not a controlled same-engine experiment.\n\n| MATH-500 · thinking enabled · 32,768-token limit |  |  |  | \n|---|---|---|---|\n| Model | Correct | Accuracy | Direct weight change | \n|---|---|---|---|\n| Unedited Qwen3.5-0.8B <sup>a</sup> | 35 / 500 | 7.0% | None | \n| RikiyaT / Astra edit <sup>b</sup> | 261 / 500 | 52.2% | Feed-forward layers 15 and 19 | \n| Output-head probe graft | 276 / 500 | **55.2%** | Three output rows | \n\nThe observed lead is three percentage points. There are **46 questions only our sample gets right and 31 only the reference gets right**. An exploratory paired bootstrap gives a difference interval of −0.4 to +6.4 points; descriptive McNemar *p* = 0.110. This does not establish superiority or equivalence.\n\nOur mean total output is **15,829 tokens**. On the 20 questions with published reference sample text, ours averages 15,340 versus about 21,638 retokenized reference tokens: **29.1% fewer on that subset**. That is not a 500-question token-saving estimate.\n\n GSM8K and IFEval are follow-up evaluations. Their final results are not included in this release. [Frozen results](https://github.com/nextgen-user/think-graft/tree/main/qwen08/results)\n\n## Watch the output, including the failures.\n\nThese are saved generations, replayed by token position—not live inference. The examples include a shorter success, an extra correct answer, and two failures. The complete 500 outputs are downloadable.\n\n## Read a signal. Edit its output row.\n\nBefore choosing a token, the model produces a vector called the *hidden state*. The output layer turns that vector into a score for each token. One row scores `</think>`; other rows score `Wait` and  `Wait`.\n\nA *linear probe* is a classifier that scores this state with a dot product. If its weights are *g*, its score is *g*<sup>T</sup>*h*. Adding the probe to an output row builds that same calculation into the token’s existing score. No separate probe runs during generation.\n\nFor this 0.8B candidate, the stop row combines a re-checking probe with a loop-boundary probe. A third probe suppresses the two Wait tokens when the state resembles looping. It is an extension of [post 1’s single-row stopping edit](https://probe-graft.vercel.app/), not the exact same one-row model.\n\n## The actual three-row recipe\n\nThe second equation applies to both spellings of Wait. Coefficients were selected using development problems, not benchmark answers. This is still a weight edit; the intervention is at the output head rather than inside the transformer blocks.\n\nAt a fixed hidden state, the logit changes are directly calculable. Top-k/top-p filtering and subsequent generation prevent an exact reconstruction of every edited trajectory from an original trace. All scores here come from live generation.\n\n## A concrete example: 25 choose 2.\n\nOne fitting problem asks for C(25, 2). The base trace calculates 25 × 24 / 2 = 300, states “Final Answer: 300,” and then keeps going to the 16,384-token fit limit.\n\nCalculation: 25 × 12 = 300.\n\nFinal Answer: 300.\n\n…followed by further checking.\n\n1. **Record the base model.** Save its final hidden states while it reads its own reasoning traces.\n2. **Label useful distinctions.** Mark paragraph boundaries after an answer appears, and repeated-line regions; use other reasoning states as negatives.\n3. **Fit the directions.** Shrinkage linear discriminant analysis finds directions separating those groups. It is a linear-algebra fit with labels, not gradient descent or base-model retraining.\n4. **Calibrate, then freeze.** Fit on 80 MATH-training questions, select strengths on 16 separate training questions, and keep the chosen weights fixed throughout the benchmark.\n\nThe output location is easy to specify: known token rows, no search over internal layers. The data work is not trivial. Collecting traces can take hours; the matrix solve is the quick part. An answer appearing in text is also an imperfect label—it does not prove the model has finished checking it.\n\n## Same problem. A different place to intervene.\n\n[RikiyaT’s Astra experiment](https://rikiyat.github.io/blog/weight-space-programming/) removes a loop-associated direction from two internal feed-forward matrices. [Nathan’s LP follow-up](https://nathan.rs/posts/weight-space-lp/) finds a related sparse internal edit. Here, the transformer computation is kept intact and the final token scores are edited directly.\n\nBoth families use activations and can be state-dependent. “His is constant; ours is conditional” would be too broad: Nathan measured a near-constant effect for a particular LP edit. An ordinary checkpoint with no runtime hook is also a feature of his method, not unique to this one.\n\nThe practical advantage I’m testing is a more mechanical recipe: record final states, fit a classifier, write a few known output rows. There is no search over which internal layers to alter. The similar observed scores make that recipe worth testing further; fewer edited weights alone do not guarantee fewer side effects.\n\n## Earlier 4B, DeepSeek and agent observations\n\nThe earlier single-row method was tested separately on Qwen3.5-4B and DeepSeek V4.1 Flash. On eight paired 4B math questions, both models got 8/8 while total tokens fell 54%. DeepSeek’s six shared solved questions used about 40% fewer total tokens. Agent examples showed selective thinking around planning, errors and new information.\n\nThose are small exploratory studies with configuration caveats, not proof of universal transfer. Our local DeepSeek internal-layer interventions failed to shorten reasoning, but they were not a controlled reproduction of RikiyaT’s exact method. I cannot conclude his method fails on DeepSeek.\n\nThe algebra does not depend on a particular model size. Useful signals, calibration and quality do. Each model needs its own fit. [Read the original experiments and limitations.](https://probe-graft.vercel.app/)\n\nRelated work already includes [probe-guided stopping](https://arxiv.org/abs/2504.05419), [activation directions controlling reasoning length](https://arxiv.org/abs/2506.08390), and [NoWait token suppression](https://arxiv.org/abs/2506.08343). The contribution here is an output-weight implementation and its measurements, not the invention of early stopping.\n\n## Ending thinking is not enough.\n\nThe largest failure became clear only after the full run. The edited model closed thinking in 163 of the 165 capped cases, then continued in the answer. Across 407 responses it emitted **1,348,952 extra closing tags**, around 17% of all generated tokens. The model has no hard phase gate; its stop boost can remain active after the first close.\n\nThe grader does not automatically reject capped runs. It extracts the last boxed answer after the last close, or from the full response if thinking never closed. An audit found **no correct boxed answer anywhere in the 165 capped responses**. Unboxed candidates may still go uncredited under this protocol.\n\nA cheap 8-gram detector, applied to saved outputs, flagged all 165 capped runs and 206 of 335 completed runs. That is alert coverage, not independently annotated precision/recall. It does not establish that forcing a stop at the alert would produce the right answer.\n\n## Completed checks, open controls, and selection caveats\n\n- **Done:** complete MATH-500; separate fitting/development IDs; frozen weights; exact tensor comparison; sampler checks; capped-answer audit; a prefix-only repetition diagnostic; published raw results.\n- **Still missing:** tuned penalty/sampling controls, avg@8 or larger repeated draws, independently labeled detector precision/recall, and systematic batch-invariance checks.\n- Presence penalty 1.5 is enabled. That is one setting, not a calibrated penalty sweep. A fixed seed does not remove serving noise.\n- The development set was reused for model selection. A fresh 16-question check scored 8/16 for this candidate versus 9/16 for the previous graft. This negative result is retained.\n- The earlier p = 0.0078 referred to token reductions on eight exploratory Qwen-4B pairs, not an accuracy improvement or a test that excludes sampling and serving noise.\n- Qwen’s tied embedding/head requires a new output matrix: roughly 0.51 GB extra. There is no extra transformer pass, but it is not zero memory overhead.\n- The current three-row model can change non-stop token choices. It must not be described as the old “only one logit changes” model.\n\n## Reproduce this run.\n\nThe GGUF is an experimental artifact, including the failures above. Start with unedited Qwen3.5-0.8B. All 299 original tensors were checked byte-for-byte; only the separate output head differs. The portable rebuild matches the evaluated file’s SHA256.\n\n```\nhf download vaibhavrpi/Qwen3.5-0.8B-probe-graft-GGUF \\\n  Qwen3.5-0.8B-BF16-probe-graft.gguf --local-dir .\nllama-server -m Qwen3.5-0.8B-BF16-probe-graft.gguf \\\n  -ngl 99 -fa on -c 278528 -np 8 -b 256 -ub 256 --jinja\n```\n\n## Sampling, seeds, environment and scoring\n\n| Sampling | temperature 0.6; top-p 0.95; top-k 20; min-p 0 | \n|---|---|\n| Penalties | presence 1.5; frequency 0; repetition 1.0 | \n| Seed & samples | Seed 0 for every request; one sample per problem | \n| Budget | 32,768 new tokens; thinking enabled | \n| Sampler order | penalties → top-k → top-p → temperature | \n| Serving | 8 slots × 34,816 context; batch and microbatch 256; flash attention; all layers on GPU | \n| Hardware | RTX 4060 Laptop, 8 GB; single GPU | \n| Software | Ubuntu 26.04; CUDA 13.2.78; driver 595.71.05; Python 3.14.4 generation, 3.12.12 fitting/scoring | \n| Engine source | `3549c274dc22001d9118713fa56a10085b85de3b` , plus included sparse presence-penalty optimization | \n| Base revision | `2fc06364715b967f1860aea9cf38778875588b17` | \n| Scoring | math-verify 0.9.0; IFEval strict prompt-level checks | \n\nThe llama.cpp penalty window is 33,792 tokens and includes the prompt; the reference vLLM penalty covers generated tokens. The sparse implementation preserves penalty arithmetic; numerical checks are not a claim of zero serving noise.\n\nFor math, the exact user suffix is `\\nPlease reason step by step, and put your final answer within \\boxed{}.` Dataset revisions, package lock, environment manifest and reconstruction commands are in the repository. All benchmark generation is live; playback is for inspection.", "url": "https://wpnews.pro/news/does-weight-space-programming-need-to-touch-the-hidden-layers", "canonical_source": "https://probe-graft.vercel.app/2/", "published_at": "2026-11-16 19:35:14+00:00", "updated_at": "2026-11-16 19:54:29.638621+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "machine-learning", "artificial-intelligence"], "entities": ["Qwen3.5-0.8B", "MATH-500", "Astra", "RikiyaT", "Nathan", "GSM8K", "IFEval"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/does-weight-space-programming-need-to-touch-the-hidden-layers", "markdown": "https://wpnews.pro/news/does-weight-space-programming-need-to-touch-the-hidden-layers.md", "text": "https://wpnews.pro/news/does-weight-space-programming-need-to-touch-the-hidden-layers.txt", "jsonld": "https://wpnews.pro/news/does-weight-space-programming-need-to-touch-the-hidden-layers.jsonld"}}