Illustration generated with AI.
Researchers at MIT CSAIL have shown something surprising: a powerful AI model can inspect the weights inside other neural networks, make targeted changes to them, and even write weights for small networks from scratch.
In one experiment, the team asked GPT-6 Astra to investigate Qwen3.5‑0.8B, a small open-weight language model that often got stuck repeating itself. Astra used Python, examined practice runs and changed 131,072 of its roughly 800 million weights. In the researchers’ tests, the edited model finished more answers and scored better on math and instruction following.
The team calls this approach weight-space programming. Their research post describes a set of early experiments, not a general solution for understanding or improving every model. But the possibility is hard to ignore: the numbers inside a neural network may become something we can investigate and change with help from AI.
For developers, that could eventually mean smaller models adapted to specific tasks, running locally or on cheaper infrastructure. To see why, we first need to look at what those numbers actually do. When we write software, we can usually follow a function, find a condition, and understand why a particular input takes a particular path. Neural networks are different. Their behavior emerges from layers of calculations controlled by learned numbers called weights.
Illustration by the author. The network and numbers are simplified examples; an LLM has a more complex architecture.
In this picture, the lines represent connections between parts of the network. Each connection uses a weight, a number that affects how a signal contributes to the next calculation. The values shown are examples, not actual weights from the MIT experiment.
A weight is a number used in a calculation. In a very simple artificial neuron, each input is multiplied by its weight. The results are added together with a bias, and the neuron applies a function to that sum. Change a weight, and the same input can have a different effect on the output. Think of weights as adjustable knobs in a calculation, with a crucial difference: in a real model, a knob rarely controls one clearly named behavior.
An LLM performs a much richer sequence of calculations. It converts pieces of text, called tokens, into numerical representations, processes them through many layers, and produces scores used to choose the next token. It repeats the process as the answer grows. The weights are the learned numbers used in those transformations. The model’s behavior comes from how many weights work together across the network.
During training, the model makes predictions, measures how far they are from the training target, and gradually adjusts its weights. Those adjustments accumulate across enormous numbers of examples. The resulting weights are available to inspect in an open-weight model, even though the full training data and process are not necessarily public.
We can inspect those numbers, but a weight rarely comes with a label explaining what it means. A behavior can be distributed across many components. Even with the architecture and weights available, tracing a model’s decisions is difficult.
That is why the inside of a language model can feel like a black box. Not because it is literally inaccessible: researchers have spent years studying its internal features and circuits. The hard part is connecting the numbers to a human-readable explanation of behavior.
Training typically changes weights by using gradients to guide many updates. The MIT team explored a different route: let a general-purpose language model reason about a network and directly write some of its weights.
Think of it as investigating a compiled program without having its original source code. An experienced engineer may infer what a piece does and patch it, but the raw instructions are not a neat map of the original design. The analogy has limits: weights are numerical parameters in distributed computations, not instructions with a one-to-one translation into source code.
The team began with networks much smaller than a modern LLM. They trained four networks to play simplified grid versions of Breakout, Pong, Pac-Man and Freeway. Each used the same architecture, input layout and possible actions.
The researchers gave Astra the weights as text, described the inputs and actions, and asked which game each network was trained to play. They did not provide the game names as options or let it use tools. According to the team, Astra named all four correctly.
One clue was where certain first-layer weights were concentrated. In the Breakout network, the weights for the player lay along the bottom row of the grid, consistent with a paddle. That is a useful kind of explanation: a pattern in the numbers connected to something the game actually does. It does not mean every weight in the network became understandable.
The team then tried a repair. A car controlled by a small trained network kept driving into a ramp and getting stuck. Astra received the weights, a description of the scene and one failed run. In one response, without tools, it proposed a change affecting 3.5% of the weights. The researchers report that the edited policy backed up before accelerating and succeeded in all 1,024 evaluated episodes. The original had failed in 256 episodes.
There was also a test of writing a network from scratch. Given the rules, game code and network layout, Astra generated weights for three simplified MinAtar games without training or playing them first. Its Breakout network scored 517.4 on average, compared with 25.3 for the team’s PPO baseline trained for 500 million steps. That comparison is for this particular game setup and baseline, not proof that written networks generally outperform trained ones.
These are small, structured environments with known inputs, outputs and rules. Still, they show three distinct things: reading a clue, modifying a behavior and writing a small network to perform a task.
Qwen3.5‑0.8B has roughly 800 million parameters. Its weights do not fit into a prompt like the small game networks did, so the researchers changed the setup. Astra worked through a Python session with access to the weights and was given a budget of up to 50 batches of practice runs. A batch could contain multiple problems. It could inspect the results and make further edits, but it could not use gradient descent or train the model.
The issue was visible in the practice runs: 85 of the first 96 answers reached the 32,768-token limit, and the researchers observed the model getting caught in reasoning loops. Astra investigated a direction associated with repetition: a pattern across many numerical dimensions, rather than one weight labeled “loop.” It identified two layers involved in writing that direction and changed 64 selected output columns in each. The released experiment logs record the analysis and the resulting edit: 131,072 weights, or about 0.016% of the model. This is the edit produced with the 50-batch budget below.
Under the researchers’ evaluation settings, the version produced with the 50-batch budget changed the results like this:
Author-created chart based on the researchers’ experiment 8*. The final measure is the cutoff rate on MATH-500, where lower is better; the first three are scores, where higher is better.*
MATH-500 and GSM8K test mathematical problem-solving; IFEval tests whether responses follow specified instructions. The test sets were separate from the practice problems.
Both versions used thinking mode, a temperature of 0.6 and a 32,768-token limit. The Qwen model card says this model defaults to non-thinking mode and recommends a temperature of 1.0 for thinking-mode text. It also warns that loops can occur even with recommended settings. The comparison shows what happened under the researchers’ configuration; it does not tell us whether changing the unedited model’s settings would have achieved a similar result. Reaching the token limit is a useful signal, though it does not by itself prove that every cut-off answer was looping.
The results are striking, but the last row helps explain the others. This was not simply a model suddenly learning mathematics. The edit removed an obstacle that stopped many answers from finishing. It also did not eliminate looping entirely.
The researchers checked this distinction with another model, Gemma 3 270M, which rarely had the same looping problem. Its mistakes included ordinary arithmetic and reasoning errors. In that experiment Astra could not establish a meaningful improvement in math by editing the weights.
There was another useful failure. Astra could not identify which of two Gemma models had been trained to agree with users just by examining their weights. It succeeded on three model pairs after it was allowed to inspect attention patterns from chosen inputs. The box had become easier to investigate, not transparent.
Imagine a wholesale distributor receiving purchase orders from hundreds of business customers. Each customer uses different document layouts and wording. One order lists a delivery date on the first page; another updates it in an amendment several pages later. Before an order enters the distributor’s system, someone has to find the products, quantities and final delivery date.
An AI model could read the document text and turn it into a structured order for a person or another system to check. This is more than choosing a category: the model has to connect details across a document and avoid treating an earlier date as the final one. For scanned documents, the text would first need to be extracted, for example with OCR.
Suppose the company processes 200,000 documents per month and uses GPT-6.1 Sol through an API. For an illustration, assume each request averages 5,000 input tokens, including instructions and extracted text, and 500 billed output tokens, including any reasoning tokens. At OpenAI’s published standard rates on November 5, 2026, that is $2 per million input tokens and $10 per million output tokens. Without caching or batch discounts, the monthly model cost would be approximately $3,000: $2,000 for input and $1,000 for output.
Now imagine the company adapts a small open-weight model specifically for its purchase orders and can run it on its own server. If it handles the same volume with sufficient accuracy and the server and ongoing operation cost an assumed $800 per month, the difference would be $2,200 per month, or about 73% of the API cost in this example.
Illustrative scenario: the API calculation uses the stated token assumptions and published Sol rates; the small-model cost is assumed. This is not a result of the MIT experiments.
Those figures are a way to understand the tradeoff, not a measured saving. The token usage, workload and $800 hosting estimate are assumptions. A cheaper hosted model, caching or batch pricing could change the comparison; OCR, review of uncertain documents, setup and adaptation also need to be accounted for. A small model that extracts the wrong quantity or misses an updated date could cost the business more than it saves. The opportunity only becomes real if it meets the company’s quality and throughput requirements.
Today, the team could try better instructions and examples, constrained output where supported, or fine-tuning. LoRA, for example, trains a relatively small set of additional parameters while keeping the original weights fixed. Compare the options on held-out orders, including amendments and unusual layouts, and measure field accuracy, latency and total cost.
If weight-space programming becomes reliable, a future tool might offer another option: investigate a recurring error in the smaller model and suggest a targeted change to its weights. The developer would still test the result, check for regressions and be able to roll back. The MIT team did not demonstrate purchase-order extraction or compare its cost against existing methods. This is a possible direction for the research, not a ready-made way to deliver the savings above. The from-scratch experiments point in a second direction. For a narrow task with well-defined inputs, outputs and rules, a developer might eventually describe a small network and have an AI construct its weights. The game results make this idea concrete, but they do not show that an AI can build a dependable network for an arbitrary business process. Sometimes ordinary code will be simpler and easier to test.
Before relying on a patched model, I would want a representative test set, checks for regressions outside the target behavior, a versioned original and a way to roll back. I would also compare the effort and total cost with prompting, fine-tuning and conventional software. The post reports research experiments and released logs, not a ready-made production workflow or a measured saving for this use case.
What interests me most is not that one model became better at a benchmark. It is the possibility of a new conversation about the model itself.
Instead of only asking, “What answer did you give?”, we may become better at asking, “What part of this network contributed to that behavior, and what happens if we change it?” The answer would still need experiments and independent verification. The research does not give us a complete explanation of an LLM, and its failures show how far that remains.
But a small, testable explanation is a meaningful beginning. If future tools can connect more of those explanations to reliable edits, developers may gain another way to adapt small models for specific work and understand why those models behave as they do.
The weights are still numbers. This research suggests that, for some carefully chosen problems, we may be getting better at asking what the numbers are doing.
AI Just Read and Rewrote the Weights Inside Another AI was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.