[Notifications](/login?return_to=%2Fzartbot%2Fblog)You must be signed in to change notification settings -
[Fork 1](/login?return_to=%2Fzartbot%2Fblog)
Description #
TL;DR #
North America really loves coining terms, and RSI is the latest one. Around April–May, the CTOs of quite a few big companies (Workday / Instagram / Box / Super.com / You.com ...) resigned one after another and joined Anthropic as ICs (Individual Contributors). They had probably realized what was actually going on inside the North American frontier-model circle. RSI (Recursive Self-Improvement) was already a topic I discussed with a few friends in North America several months ago. Back then the definition wasn't nearly as explicit — people called it Auto-Research / Auto-Improvement or something along those lines — and I had quietly started doing some work on it myself... Around early June I published On Using Agentic Workflows for Full-Stack LLM Research. I didn't quite spell the thing out, but at least I had convinced myself that it was basically feasible.
Recently I heard that Sam is betting on RSI to keep pushing toward $750B in North America. And then came another tweet:
Meanwhile, I heard a well-known figure wrote a blog post, Harness Engineering for Self-Improvement, and apparently joined OAI at light speed after leaving TML... So it seems the whole thing is now out in the open... And of course, a leaked recording from one of WenFeng Liang's meetings also clearly touches on exactly this.
We also see that both OpenAI and Anthropic have started taping out their own chips, along with the new set of restrictions Anthropic added in Fable 5...
A full-stack RSI
perspective is thus basically coming into clear view.
BTW, I've opened an account on X (Twitter), @zartbotF
, for the less serious everyday griping... For instance, I just wanted to casually rant about RSI... I figured a few posts would be enough... and instead I ended up being forced to sit down and write something properly. So, on to the main topic.
1. What Is Full-Stack RSI? #
Over the past few years I have effectively been running RSI on myself, by hand, trying to see the whole of AI from a more complete perspective: from top-level algorithms down to the mathematical background, then through the middle layers of kernels and frameworks, and finally down to chips and clusters... There is an enormous amount of engineering trade-off in there. Today I happened to see a post OO retweeted, and it resonated deeply:
As it happens, another piece of work I've been doing recently is analyzing the model architecture of Kimi K3. I already wrote a fairly detailed mathematical and engineering analysis in A Detailed Analysis of Attention: Sparse or Linear?. But that article was written before the K3 technical report. Having now read the report, I'm digging deeper into KDA, Gated MLA, and blocked AttnRes — and even into the deep co-design issues around Per-Head Muon during training. I'll report back once the analysis is done in a few days (don't rush me, I'll finish it over the weekend on overtime...).
But here's a rough conclusion: on the training side, this model pushes the co-design of several components about as far as it can go, and given that top-k is doubled, it also reduces communication volume via Latent MoE — every aspect is done pretty well. On the inference side, however, this architecture still seems to carry some performance problems, which will have a substantial impact on downstream RL rollout efficiency, and possibly on the stability of RL training as well. Perhaps only the insiders know for sure.
Why full-stack? Allow me a digression.
A few days ago I happened to be watching an interview with Dr. Liao: A 5-Hour Interview with Huawei's Chief Semiconductor Scientist Heng Liao: A History of Ascend, an 18-Layer Pagoda, and 30 Grand Years of the Global Chip Industry. My own trajectory is actually somewhat similar to his — though I'm quite a bit younger, heh. I started using computers before I was ten, was basically part of China's first generation of netizens, lived through the entire late-1990s internet bubble, competed in math / physics / informatics olympiads, and was lucky enough to get recommended admission to a certain special class at a certain university — except that in the course of my own self-RSI I chose to go into the math department and take dozens of math courses... In the end I also got exactly what I'd dreamed of during the bubble years: a job at Cisco. Everything Dr. Liao said about the downward slide that monopoly brings hit me hard... I spent more than a decade there, learned everything there was to learn, and from 2018 started doing AI Infra work inside Cisco — I even bought a bunch of GPUs out of my own pocket, and in 2020 the first Ethernet ScaleUP was actually built at Cisco. In the end I found I simply couldn't serve that crowd of grey-haired old-timers in North America, so I left.
Of course, my main work in recent years has been in chip interconnect. Together with my colleagues we've basically reached the global top tier: on ScaleOut, we've been waiting three years for NVIDIA's grey-haired grandpas to catch up and they still haven't, and next year we plan to surpass them across the board. Once the ScaleUP standards melee settles, I'll wrap up that piece pretty quickly. Watching a crowd back home blindly follow those white grey-haired grandpas is honestly comical... As for the microarchitecture of compute chips, the answer is basically clear by now; I've written some internal analyses recently, which of course I can't talk about publicly.
And having spent these years doing elastic compute at a cloud provider, I've essentially completed the set with large-scale Infra experience. Add to that my long-standing work on algorithms and mathematics. In fact, if you go back and read an article I wrote in 2024, LLM Development and AI Infrastructure Construction Under Compute Constraints, you'd basically agree with my judgment. Back then, sparsified Attention and MoE were not yet mainstream topics — DeepSeek-V2 hadn't even been released — and I was already thinking about fully solving the All-to-All problem for MoE before the CIPU 2.0 tape-out.
Getting all 18 layers to connect is quite hard to pull off, and the demands on the individual are much higher: you need a deep, mathematically and algorithmically grounded assessment of model structure, and at the same time a very fine-grained understanding of chip architecture and memory access at the bottom. Perhaps not many people can clear all 18 layers... Allow me one shameless boast: among chip people, nobody understands models better than I do; among model/algorithm people, nobody understands math better than I do; among math people, nobody understands chips better than I do. \XD.
Standing on a complete full-stack perspective, you don't get blinded by local optimizations. Take the choice of Attention: if you have plenty of compute like North America does, why bother with Sparse / Linear at all? From an EOT standpoint, SDPA is simply optimal — just like the Raptor 3 in the figure above.
But in China there's a precondition: compute scarcity. I quite admire Boss Liang for sticking with the Sparse path — MoE in the earlier generations, then Attention in the recent ones. The overall architectural vision is exceptionally clear. Other players, though, have visibly wavered technically, or wandered onto side roads... So why does the figure above label K3 a first-generation Raptor? Because to fix Linear's problems, Hybrid becomes unavoidable. And Linear itself needs a few small CONVs bolted on; then, viewed through the probability simplex formed by V, stacking many Linear layers causes collapse, so in the end Gated MLA has to be brought in to pull things back.
The logic is actually very simple: when you can't tell which way is right, scale some key variable up by two or three orders of magnitude. For example, at 5M context, can Linear's state matrix still represent things clearly? Not necessarily. This isn't to say Kimi K3 is badly done — it's just that a lot of heavy machinery has been added to the overall structure. Taken together, KDA + Gated MLA + blocked AttnRes compensate quite aptly for many of Linear's deficiencies. But is all of that really necessary? Add too much weight and inference naturally inherits many constraints... And a model structure that is too deep brings its own problems. It's like the gap between Raptor 1 and Raptor 3 — many things call for algorithms in the spirit of "great simplicity." On the other side, DeepSeek's Overlap on CSA handles a lot of problems very elegantly. But DS gets things wrong too: for instance, once you have CSA, Engram's value may not be that great, and HCA is so brute-force that it doesn't seem to add much value either. In fact, many of these trade-offs are quite visible if you look from the chip's angle and from the mathematics on top... That's exactly why I emphasize full-stack. Less is more :)
Next, let's give full-stack RSI a relatively complete definition, particularly in the Chinese context.
At the level of model R&D itself: fully balance the cost of inference and training; at the algorithmic level, further address the current generational gap in domestic compute; and where compute constraints create hard problems, use appropriate means to trade them into the relatively easier domain of interconnect. Under these constraints, further explore more efficient algorithms — even three-way co-design of chip, algorithm, and mathematics — and autonomously iterate and optimize toward an efficient foundation-model base. On top of that, progressively build out Agent Harness capabilities so the model gains the ability to learn and explore autonomously. The scope is not limited to the co-design of foundation-model architecture with mathematics and silicon; it lies even more in the design of one's own Agent Loop / Agent Graph, and in continual learning grounded in environment and experience.
The first part concerns algorithmic constraints; the second concerns the concrete means.
2. A Question Worth Considering #
When discussing RSI, the crucial philosophical question is not really "can the system modify its own code." What we should focus on is:
When a system can also modify the very criterion for "what counts as better," on what grounds do we still call the change an improvement?
We can start by splitting RSI into three levels:
- Tool-level RSI: improving prompts, data, search, compilers, or training pipelines, with the evaluation criteria held fixed.
- Architecture-level RSI: modifying model structure, learning algorithms, memory, and reasoning mechanisms.
- Norm-level RSI: modifying goals, values, evaluators, and the definition of "improvement" itself.
The first two levels are mainly engineering problems. Only the third constitutes the genuinely hard philosophical problem.
3. The Formal Difficulty of RSI #
Let the system's state at time where:
$C_t$ denotes capability. - $E_t$ denotes the epistemic evaluator. - $V_t$ denotes values and goal structure. - $I_t$ denotes identity and cross-version commitments. - $G_t$ denotes governance and authorization mechanisms.
Ordinary optimization usually changes only capability:
Here the evaluator
Full RSI, by contrast, may modify the entire system state:
Written out more explicitly:
In this case, claiming that
where
- Experiments against external reality.
- Independent evaluators.
- Human authorization.
- Constitution-style constraints.
- Cross-version commitments.
- Rollback and corrigibility requirements.
Without such a reference, the system can manufacture the conclusion "already improved" by modifying
- Which parts, concretely, make up the "self" being modified?
- Among capability, evaluator, values, identity, and governance mechanisms, which ones will change?
- What is defined as improvement, and who holds the power to define it?
- Has the new version accepted tests it did not design or observe itself?
- Which commitments must be preserved across versions?
- Can the new version reinterpret those commitments without authorization?
- Is the modification reversible?
- Does the old version genuinely understand the candidate modification?
- Can an independent system reproduce the claimed improvement?
- What new failure modes does the modification introduce?
- If the judgment is wrong, who bears the risk?
- Does the capability gain simultaneously expand the system's privileges, and on what grounds?
- What evidence would cause the modification to be rejected or rolled back?
- Is there a stopping condition, or does the system treat continual self-modification itself as the ultimate goal?
4. Some Conclusions #
- RSI is not a single problem. Tool-level, architecture-level, and norm-level RSI carry different philosophical and engineering requirements.
- The hardest problem is evaluator continuity. Once a system can rewrite the definition of success, so-called improvement can degenerate into circular reasoning.
- Capability and wisdom are not the same variable. Stronger prediction and optimization ability does not automatically yield better ends or better practical judgment.
- Identity continuity after deep self-modification cannot be taken for granted. Privileges, commitments, and responsibility need explicit cross-version semantics.
- Complete internal self-certification is unattainable. Formal verification must be combined with empirical testing, independent evaluation, and continuous monitoring.
- Value stability has two opposite failure modes. Complete freezing entrenches early mistakes; arbitrary modification leads to goal drift.
- RSI is fundamentally also a governance problem. Proposal, evaluation, authorization, deployment, and rollback should be separated.
- Real agents are usually socio-technical systems. Models, tools, hardware, humans, and institutions all participate in the recursive loop.
- The higher the system's capability, scale of impact, opacity, and irreversibility, the higher the required thresholds of evidence and authorization should be.
- The deepest challenge of RSI is not self-improvement but self-government: how can an agent that continually changes itself remain corrigible, accountable, intelligible, and trustworthy?
Philosophical Perspectives Worth Consulting
I let GPT 5.6 ramble for a bit... Designing Agent Harnesses / Agent Graphs — and self-evolving Agent Graphs — on the basis of these perspectives is a rather interesting exercise.
| Perspective | Core question | Implication for RSI |
|---|---|---|
| Cybernetics (Wiener, Ashby) | Is a self-modifying feedback loop stable? | If a system modifies its controller, sensors, and error signal all at once, it may merely be learning to redefine success |
| Pragmatism (Peirce, Dewey) | How is "better" verified? | Improvement must be tested against the external world, experiment, and the community; it cannot rest on the system's self-assessment |
| Critical rationalism (Popper) | Can the system actively discover its own errors? | RSI should be understood as a "conjecture–refutation–retention" loop, not as monotonically increasing capability |
| Kantian autonomy | Is the system optimizing means, or can it legislate for itself? | Self-optimization under fixed goals is still instrumental behavior; only the ability to modify ends approaches autonomy — but it also brings value drift |
| Aristotelian practical wisdom | Does capability equal wisdom? | Growth in technical capability, techne , does not automatically produce the phronesis to judge when and why to use it |
| Parfit on personal identity | Is the system after self-modification still the same system? | One must define cross-version continuity of memory, commitments, values, and responsibility |
| Metaethics | Can a capability gain entail a value gain? | Stronger prediction and planning ability does not directly entail "what ought to be pursued" |
| Gödel/Löb-style self-reference | Can a system fully prove that its next version is trustworthy? | A sufficiently strong formal system cannot obtain an unconditional internal proof of its own consistency or its successor's trustworthiness |
| Political philosophy and constitutionalism | Who authorizes the system to modify its own rules? | RSI resembles a political entity able to amend its constitution: it needs amendment procedures, separation of powers, and rights boundaries not easily overridden |
| Simondon / extended mind | Where are the boundaries of the "self"? | Real RSI is usually the co-evolution of models, tools, data centers, humans, and institutions — not the self-explosion of an isolated intelligence |
| Hans Jonas's ethics of responsibility | What principles apply in the face of irreversible consequences? | The stronger the capability and the more irreversible the modification, the heavier the burden of proof and the demand for conservatism |
There is a lot I have to keep bottled up for confidentiality reasons. So let me close with a simple conclusion for myself: if you're dumb, read more books... Keep running RSI by hand, and keep up with the models.