As everyone knows, training a model is an extremely expensive and time-consuming process, even if you only do post-training and not pre-training.
Please improve an AI that is bad at math without any additional training
When given this task, we would normally think about adding more training data, fine-tuning, reinforcement learning, or improving its reasoning process at inference time.
However, a new research project has explored a completely different approach.
The idea is to let an LLM examine the internal structure of another neural network, figure out which weights need to be changed and how to change them, and then directly modify those weights.
An article went viral on X on November 2, mentioning that researchers at MIT CSAIL had GPT-6 Astra drastically modify the weights of the Qwen3.5–0.8B algorithm. The changes were minimal, only 0.016%.
But the result shocked everyone: under the research team’s test configuration, the mathematical accuracy rate rose from 7.9% to 48.1%.
However, it would be too early to conclude that the AI has learned mathematical skills from scratch.
The main achievement here is that the researchers fixed a problem where the model repeatedly got stuck in its reasoning process and failed to produce an answer. By fixing this issue, they were able to unlock capabilities the model already had but couldn’t properly demonstrate.
Even on its own, this is a pretty interesting result.
If you like this topic and you want to support me: First, let’s talk about “weights” inside a model. They are a large collection of numbers that the model uses when processing input.
Changing these numbers changes how the model produces its outputs.
During pre-training, the model is fed massive amounts of data, such as text. It makes predictions and compares them with the expected results.
The training algorithm then calculates how the weights should be adjusted based on the errors and repeatedly updates them.
This entire process requires collecting and cleaning huge amounts of data, as well as significant computing resources.
After a model is trained, it usually goes through another stage called post-training to make it better at following instructions and handling specific tasks. One common method is supervised fine-tuning using examples.
Different methods use different data and training objectives, but they often still require repeatedly processing data, updating parameters, and evaluating the results.
The challenge is that you might only want to fix one specific problem, but you may not know which numbers inside the model are responsible for it.
Further training can change the model’s behavior, but preparing data, running experiments, and evaluating the results all come with costs. Even after fixing one problem, you still need to check whether other capabilities have been affected.
So, what if we could identify the source of the problem and directly modify the relevant weights?
That’s the approach the MIT team decided to explore. However, their experiments have not yet demonstrated whether this approach is actually cheaper overall.
In this research, the team conducted an experiment using a state-of-the-art large AI model (GPT-6 Astra) to directly modify the internal parameters of a lightweight, open-source AI model (Qwen).
Inside an AI model, there are enormous collections of numbers called weights (parameters). You can think of them as similar to the strength of connections between neurons in the human brain.
They essentially form the model’s internal structure, influencing how it processes information and generates outputs.
In traditional AI development, improving a model’s performance or making it better at a specific task usually involves techniques such as fine-tuning or additional training.
This means repeatedly feeding the AI large numbers of questions and answers (training data) and spending significant computing time gradually optimising its internal parameters.
However, the approach used in this research is different. No additional training was performed.
Instead, an advanced AI model directly examined the target model’s internal weights, analyzed where potential problems or bottlenecks were located, and then selectively rewrote specific values.
To put it simply, imagine a human patient. Instead of making them solve thousands of problems to improve their abilities, a skilled surgeon examines their brain’s neural connections and fixes a few specific problems directly.
According to the report, only about 0.016% of the model’s total weights were modified.
Despite such a small change, the results showed a dramatic improvement, with the math benchmark score increasing from 7.9% to 48.1%.
Rather than teaching the model entirely new knowledge from scratch, the researchers appear to have fixed an internal structural problem that caused the model to get stuck in reasoning loops and fail to produce answers.
Although this research is a major technical achievement, it also raises some serious concerns.
My biggest concern is that the performance gap between state-of-the-art commercial AI models and open-source models that anyone can modify could shrink much faster than before.
Leading AI models from companies like OpenAI and Anthropic have strict safety standards and guardrails designed to prevent them from generating harmful information.
On the other hand, open-source models can run locally and offer users much more freedom to use and modify them.
Until now, bringing an open-source model up to the level of the most advanced AI systems has required enormous resources.
This created a certain barrier that made it difficult for individuals or organisations to independently develop and maintain highly capable models.
However, if advanced AI models can directly identify and fix the weaknesses of open-source models, we might eventually see powerful AI systems being developed in less time and at a lower cost, potentially without the same safety restrictions.
Another concern is that directly modifying a model’s internal weights can be used for both good and bad purposes.
When used responsibly, this technology could improve safety and fix existing problems. But in the wrong hands, it could also be used to completely disable the safety mechanisms built into a model.
This research introduces a new possibility for improving AI models.
In the future, instead of spending huge amounts of computing power on retraining or post-training every time we want to improve a model, we might be able to use another AI to identify which weights need adjustment and modify them directly.
If this approach works at scale, developers could use it to fix problems in existing models. Going one step further, could we also directly add new capabilities by modifying the weights? That’s another direction the research team wants to explore. However, this experiment focused on fixing an existing problem. Astra did not perform gradient-based training. Instead, it helped Qwen overcome repetitive reasoning loops.
Identifying problems and testing the results still require data and computing resources. Whether this approach is actually more cost-effective has not yet been directly demonstrated.
The research team also mentioned RSI (Recursive Self-Improvement). The idea is that if a model can eventually examine and modify its own weights, then use its improved capabilities to continue optimising itself, AI could gain an entirely new path toward self-improvement.
In this experiment, Astra modified Qwen. The idea of an AI modifying itself and becoming more powerful through repeated cycles is still a vision for the future.
Reference :
Most Important Paper: AI Can Read & Write Weights was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.