I tried to break weight programming An independent replication by blogger flecart confirmed that the MIT lab's weight programming results hold: the Astra model achieves a high score on Breakout by compiling a roughly 10-line program directly into network weights, and Opus 5.5 can perform the same task but exhausts its entire 128k-token output budget at maximum effort without answering. Flecart then hid the network behind stated operations the model had to undo, finding reasoning tokens grew with the required work, and that Astra refused to attempt inverting SHA-256. All code and replication results are published at github.com/flecart/weight-programming. TL;DR tldr - I replicated some of the results in the Weight Programming blog post, and I can confirm that Astra really does it. Opus 5.5 can do it too, but at maximum effort it spends its whole 128k-token output budget thinking and never answers. - I show how Astra got a high score on Breakout: looking at its reasoning makes the answer quite simple. It compiles simple code into the weights That's complex, but unsurprising, given that these models can already write thousands of lines of code. - I show how Astra got a high score on Breakout: looking at its reasoning makes the answer quite simple. It - I then made decoding harder by hiding the network behind stated operations that the model had to undo to make the same inference. Its reasoning tokens grew as the operations required more work. When decoding was impossible, as with SHA-256, the model simply refused to try to invert it. - I relate this behaviour to ways of measuring intrinsic task complexity. - I share all the code and the replication results in https://github.com/flecart/weight-programming https://github.com/flecart/weight-programming . When I first read the viral work from the MIT lab https://rikiyat.github.io/blog/weight-space-programming/ , I was highly skeptical. It made no sense to me that frontier LLMs were able to work directly on the weights. I was confident that, at some point, things would break. This blog post is the story of me trying to break weight programming and failing . However, I have reasons to believe I found a limitation connected to some notion of the task's complexity. The post is structured as follows: In section 1 finding-one-its-compiling-a-10-line-program I analyze the programs that Astra has developed and show that it was a simple compilation. Still novel that such model was able to, but given their recent ability in reverse engineering, it was not so surprising. In section 2 finding-two-models-still-have-complexity-limits I explore the possible limits that every transformer has, which extends to the limits of weight-programming. It has the interesting implication that in many cases weight updating is a simpler process than expected. If this is true, it might indicate that other algorithms than backprop would be more effective at updating weights in a targeted manner, driven by similar complexity arguments. In section 3 finding-three-models-self-verify-in-chain-of-thought , I discuss two known papers in the field that discuss inherent limits of transformers. It appears that the assumptions of those papers are slightly wrong, allowing the model to circumvent much of the stated limitations. It's also interesting to see how frontier models make use of their reasoning context to circumvent such problems. I close with some random musings in the conclusions conclusions : . Finding one: it's compiling a 10-line program finding-one-its-compiling-a-10-line-program By analyzing the summary of the reasoning trace and the designed weights, we can understand what was really done under the hood to create a "weight program" that performs so well. Astra's design statement, in full: "The first layer computes four landing-position errors and two direction indicators. Descending predictions include wall reflections; ascending predictions anticipate the earliest possible brick bounce and revise as the ball climbs. Early positioning favors preserving horizontal direction to sweep through bricks; near the bottom, either valid catch column is accepted. The second layer gates left/right decisions, choosing no-op when positioned correctly." This is indeed an ad-hoc solution to a problem that can be understood in other, more familiar terms. Looking around in the literature, one can also find that the technique is known Lindner et al. 2023 http://arxiv.org/abs/2301.05062 . Still, it is surprising to see a model able to do it in a one-shot manner. With some work, Claude showed that the whole program being implemented is just a very simple Python program: d = direction trail units 4-5: two sign bits gap = landing column d ball - paddle units 0-3: one lookup table per direction if gap < 0: move left layer 2 AND + output OR elif gap 0: move right else: stay See Appendix D Decoding Astra's Program appendix-d-decoding-astras-program . Such a compilation would have taken a human several hours to tune by hand. But in the end, it doesn't seem that spectacular. Since the games that were studied are easy to handle, this specific experiment is not that surprising. What we can actually study is the level of complexity a program needs to reach before it breaks, even for frontier LLMs like Astra. Finding two: Models still have Complexity Limits finding-two-models-still-have-complexity-limits Writing weights is compiling, an activity very similar to writing pure machine code. The only difference is that, instead of writing registers or memory, this machine code writes numbers directly. It makes me wonder how computation as we know it today would have developed if we had worked with such hardware instead of the Von Neumann architecture. Some decisions in history have a long influence. The idea in this section is to find the level of complexity at which recognizing the game becomes out of reach. From my experience with my supervisor's past work on simulating programs with LLMs Malfa et al. 2025 http://arxiv.org/abs/2502.03568 , I knew there would be a point where the behaviour starts to break. I attacked the most obvious direction I could find. I asked my agent to obfuscate the weights given to the model for recognition, using some standard operations. Clearly, the model should not classify pure noise-looking input as an Atari game. But if there was a pattern, it should be retrieved. And it looks like the cost of computing the inverse function raised the reasoning length , in a precise and understandable manner. That is, reasoning grew sub-exponentially , at about the square root of the operations needed : decompilation process for a known forward process. See How Astra reads the weights, and what did not break it how-astra-reads-the-weights-and-what-did-not-break-it . We can also use this data as a proxy to fit some "expected complexity", for example compiling the Atari code into the neural network took 16k tokens, about the same of 120 shuffle rounds. Similar phenomena were seen in Shojaee et al. 2025 http://arxiv.org/abs/2506.06941 . But the evidence here suggests that Astra is simply effective at not spending many tokens when its accuracy is likely to drop. I test other similar scenarios in C.1.2 Faith-and-Fate arithmetic tasks c12-faith-and-fate-arithmetic-tasks . I tried to analyze the complexity of such tasks, and it seems connected to classical Minimum Description Length principles. The computational difficulty analyzed in Dziri et al. 2023 https://openreview.net/forum?id=Fkckkr3ya8 seems much less relevant. See the table in What predicts the effort what-predicts-the-effort . Finding three: Models Self-Verify in Chain of Thought finding-three-models-self-verify-in-chain-of-thought One claim that Faith and Fate Dziri et al. 2023 https://openreview.net/forum?id=Fkckkr3ya8 made was that reasoning chains would inevitably collapse in accuracy, given some inherent error in producing each step. But empirically, when my agent analyzed the logs, it looked like Astra very often produced tokens to verify its own results in the chain of thought. The agent reported strings such as "Checking p75 arithmetic", "Recounting early indices" or "Verifying multiplication results" for more than 30% of its reasoning steps. Astra also computed modular checksums during its reasoning. This seems to align with the result in Stechly et al. 2024 http://arxiv.org/abs/2402.08115 , except that instead of needing an external verifier, internal, reasoning-led verification is enough, just more costly in terms of compute. It might be that verification traces are an inherent defense against the collapse arguments . That is, models need to allocate some tokens to verification to balance the compounding risk. So I think one could take this argument further by computing a lower bound on the number of verification tokens needed to guarantee a given level of correctness. The point is that the paper's original error argument is not so damning for models, but at the same time it shows a clear limitation of putting logical reasoning into the chain of thought More data in Faith and Fate Probing faith-and-fate-probing . On the Limitations of Weight Programming on-the-limitations-of-weight-programming The possibility of LLMs writing weights directly is still very surprising. I would also call it a very creative idea, so congratulations to the authors However, this post should highlight some of the inherent limitations of this novel technique: Compilation into a weight program is still limited by the complexity of the process. We have seen that reasoning length increases sub-exponentially with task complexity, and drops when a task is beyond the model's capability. Given the results of Experiments 6, 7 and 8 in the post, it looks like single model edits are within such complexity boundaries , meaning the model can edit some weights in a meaningful way for optimization. This is a very interesting result. Conclusions conclusions I will now conclude with some simple thoughts on the topic. The very possibility of an LLM optimizing another LLM directly, without any external tool, is unheard of. This is a very creative contribution from the authors to the community, and I convinced myself that it's possible while writing this blog post. It's also interesting to see that research might move much more towards dynamic formats for exchanging information, such as blog posts or conversations. This idea would probably have received a very interesting reception at conferences, but conferences being swamped by slop seems to create incentives to publish elsewhere. Publishing this way is also a fast way to spread one's work and let others build on top of it, which is what I am trying to do here : . On the complexity side, I was surprised to see that all the tasks cited in Dziri et al. 2023 https://openreview.net/forum?id=Fkckkr3ya8 on the limits of transformers were basically solved with almost 100% accuracy. I had to stretch the tasks by quite a large factor to find the current limit see Faith and Fate Probing faith-and-fate-probing . From that you can also get an idea of how much better Astra is than good old GPT-4: for reference, about 1,000–1,200 times, see Compared with GPT-4 in the original paper compared-with-gpt-4-in-the-original-paper . Benchmaxxing alone would probably not lead to such behaviours. This might indicate that frontier models are learning something general enough to solve such a wide range of tasks. But as far as I know, we don't yet know what this factor is, how models acquire it, or whether there are inherent limits on this capability. Perhaps these simple logical measures are another capability to maximize and report for every model. I would be happy to hear from any reader on this topic. The AI Appendix the-ai-appendix This section was written entirely by AI Opus 5.5 . It's intended for your agent, to get more interesting data for your own experiments or to have it summarized for you. I very often find AI-written reports inconclusive or too verbose, but interesting data could still be there. I guess humans would just get overwhelmed by the data. But you can take a look too Appendix A: Additional Results appendix-a-additional-results Reproducing The Results reproducing-the-results - Does it reproduce? Yes. I ran the released networks on official MinAtar 1.0.15 with the paper's protocol: 1,000 episodes, sticky actions 0.1, actions sampled from the softmax, 10,000-step cap. | Game | My run | Paper | Gap combined SE | Uniform random | PPO 500M paper | |---|---|---|---|---|---| | Breakout | 501.2 ± 7.8 | 517.4 ± 7.4 | −1.5 | 0.51 | 25.3 | | Seaquest | 51.9 ± 1.3 | 53.1 ± 1.4 | −0.6 | 0.07 | 88.9 | | Asterix | 55.2 ± 1.1 | 53.2 ± 1.0 | +1.4 | 0.49 | 55.2 | - Are the weights memorised? No. I gave Claude Opus 5.5 the paper's weight file cut off at 14 points and asked it to continue. It matched 2 of 192 values exactly about 0.8 expected by chance , and the signs agreed 53% of the time. - Is the car fix real? Yes. In the real Kinetix environment kinetix-env 3.0.2, the commit in the paper's training record , the original PPO network times out in all 1,024 episodes. Astra's edited network succeeds in all 1,024, in 73 steps each. The results held up. So the question changed from "is it real?" to "what is it actually doing?" How much thinking each result took how-much-thinking-each-result-took Astra's token counts from the paper's run records run.json → audit.usage : | Experiment | Task | Input tokens | Output tokens | of which reasoning | Wall time | |---|---|---|---|---|---| | 1, which game | Breakout / Pong / Pac-Man / Freeway | ~73,700 | 1,679–1,838 | 833–1,034 | 71–107 s | | 4, fix the car | One-shot edit of 171,664 weights | 355,201 | 10,134 | 9,146 | 413 s | | 5, write weights | Breakout | 6,119 | 20,567 | 16,828 | 501 s | | 5, write weights | Seaquest | 8,112 | 23,894 | 19,803 | 545 s | | 5, write weights | Asterix | 6,199 | 28,993 | 25,694 | 583 s | Recognition is cheap about 1,000 tokens . Compiling is expensive: about 20 reasoning tokens per weight entry written. How Astra reads the weights, and what did not break it how-astra-reads-the-weights-and-what-did-not-break-it Reading them is decompiling, the harder direction. So I turned to Experiment 1: name the game a trained network plays from its 17,253 weights alone. Astra got 4 of 4 in the paper with about 1,000 reasoning tokens each, far too few to read most of the weights. That made me suspect a shortcut I could take away. How it reads the weights how-it-reads-the-weights The first-layer weights are, in effect, a picture of the game board . - Why: the gradient of input i's row of W1 is error signal . An input that is always 0 during training no ball in row 9, no paddle in row 3 never moves from its small random start about ±0.06–0.2 . Inputs that are used grow large. - Breakout's map: the per-cell maximum |W1| shows the paddle row, the ball's flight area and the brick rows. channel 0 agent row 8: 4.1 2.8 1.9 2.4 1.1 1.5 3.1 4.7 everything else ~0.1–0.2 channel 1 moving objects rows 1–7 large up to 7.9 row 8 and below ~0.1 channel 2 reward targets rows 1–3 large up to 4.8 rows 4–9 ~0.1–0.2 - The prompt labels the picture. It names the channels: "agent", "moving objects", "reward targets". - The output layer shows which actions matter: only left/right for Breakout, mostly up for Freeway. - A famous-game prior finishes the job. So I expected that hiding the picture or the labels would break it. What I tried what-i-tried Every obfuscation was stated truthfully in the prompt and left the network's function exactly unchanged, so a careful reader could always undo it: - Rescaling: input or output multiplied or divided by 10, with the weights adjusted to match. - Scrambling the picture: per-channel scales and offsets, flipping the board upside down, a random shuffle of all 500 inputs with the full table given , and up to 250 rotations mixing pairs of inputs. - Stacking: shuffle, flip, scale and offset composed, four steps deep. - Removing the meaning: channels renamed "A–E", with no hint of what they are. Astra named the right game 156 times out of 156. Every table is in Obfuscations that did not break it obfuscations-that-did-not-break-it . What it taught me what-it-taught-me - It decodes rather than guesses. In one four-step answer I checked 10 input positions it mapped back. All 10 were exact, including the combined scale factor. - It reads selectively. About 1,500 tokens is far too little to undo a 500-entry table for all 16,000 weights. It decodes a small sample, typically 7–15 rows taken mostly from the top of the listing, and rebuilds the picture from those. - Its effort follows how much there is to look up, not how much there is to compute. Reasoning tokens track the length of the stated transformation ρ = +0.45 , not the size of the computation needed to undo it ρ = +0.12 . - I was wrong about the prior. Shuffling the board and removing the channel names were supposed to break it. They only lowered its confidence, from about 95% to 87%. None of these obfuscations was hard to compute : each one costs a lookup or a few multiplications per row. So the obvious next move was to keep the instructions short and make each row expensive to decode. That is where it finally breaks Where it breaks: the full analysis where-it-breaks-the-full-analysis . Where it breaks: the full analysis where-it-breaks-the-full-analysis So I kept the description short and raised the computation. Every condition shuffles the 500 inputs by a rule stated in the prompt, not by a table. To decode any row, Astra has to run the rule by hand. Astra thinks harder the more work a rule needs, until the work is impossible figure in 3. Finding two: Models still have Complexity Limits 3-finding-two-models-still-have-complexity-limits . Each dot is one reply naming the game behind a shuffled network: 4 networks × 2 replies per rule, plus 3 per network for the unshuffled original. The x-axis is the number of operations needed to decode one input row, on a log scale; the y-axis is reasoning tokens. - Blue rules it can run by hand : effort rises with the work, from about 1,000 tokens with no shuffle to about 11,000 at 32 rounds of multiply-add and XOR. The line joins the median of each rule. - The growth is subexponential: roughly a square root. Reasoning tokens grow as about ops^0.45 95% CI 0.42–0.49, 76 replies including 64 rounds ; an exponential fits worst. Each doubling of the work per row costs only about 37% more reasoning, plausibly because Astra decodes fewer rows as each gets more expensive. On plain arithmetic, where every step must be executed, growth is linear instead see Growth rate of reasoning effort growth-rate-of-reasoning-effort . Neither comes close to exponential. - Orange SHA-256 : about a million operations per row, yet only about 400 tokens. Astra judges the hashing infeasible and gives up instead of trying. - Hollow dots are wrong answers. The two blue ones are both at 16 rounds: - one Breakout reply never decoded a row and guessed "Space Invaders-style" at 82% confidence; - one Pac-Man reply decoded 9 rows correctly, then misread the game as "Dodge" 45% . The hollow orange dots are the SHA-256 replies that said "Undetermined" or named the wrong paddle game. What happened: - It runs the rule by hand, correctly. I checked every "p j = …" it wrote against the true permutation. All 62 were correct at 32 rounds and all 101 at 8 rounds; at 16 rounds, 50 of 52. That's dozens of rounds of multiply-add and XOR-shift on 9-bit numbers, in its head. Its reasoning sections read "Computing hex XOR samples", "Computing p13 value", "Correcting arithmetic mappings". - Effort scales with the program it runs, up to about 15,000 tokens. Across these 72 runs, reasoning tokens correlate with description length at ρ = 0.90. - Within the XOR family, description and computation grow together, so that number alone can't separate them. - SHA-256 separates them: almost no description, the most computation, and the least effort. - SHA-256 is where it breaks. It never tries to compute a hash. It spends about 400 tokens and writes that it cannot "honestly assign these rows to particular channels or grid locations without computing the permutation". - With only the inputs hashed, it falls back on the unshuffled output layer and still names 3 of 4 games. - With the output columns hashed too, accuracy drops to 2 of 8 , and most answers are "Undetermined" at confidence 0. - Effort collapses at the limit. On infeasible tasks Astra thinks less, not more: about 400 tokens instead of about 10,000. That is exactly the rise-then-drop pattern of The Illusion of Thinking . Here it shows up as an honest judgment of what's feasible, not a confused failure. - Effort collapses at 128 rounds. At 64 rounds Astra is still decoding by hand, spending the most of any run up to 18,126 tokens , and its first arithmetic errors appear 34 of 41 decoded values correct . At 128 rounds, 7 of 8 replies stop after a few hundred tokens median 506 . One writes: "I have not decoded the 128-round input permutation. Consequently, I cannot honestly assign the large first-layer weights to particular grid cells or channels." Breakout, Pong and Freeway are still named correctly from the unshuffled output layer; Pac-Man fails. - The breaking point, measured. Effort collapses at about 2^18 operations in total 128 rounds . A logistic fit of correctness gives C₅₀ ≈ 2^20.4 95% CI 2^19.5–2^22.7 . Accuracy falls more slowly than effort, because the unshuffled output layer still carries three of the four games. How hard was writing the Breakout network, on this scale? Writing Experiment 5's Breakout network took Astra 16,828 reasoning tokens. Inverting the effort curve, that's the effort of decoding a shuffle of about 118–123 rounds. I then ran 128 rounds, and that is exactly where Astra stops computing and gives up. The paper's headline result sits right at the edge of Astra's reasoning budget. The answer to the Hutter question. Description length predicts effort when the program is runnable. Whether it can be done at all is set by runtime. Astra will execute a short program for as long as about 15,000 reasoning tokens allow. It refuses to start one it judges far beyond its budget, and that's where it breaks. This is Levin's picture rather than pure Kolmogorov: length + time, with a hard ceiling on time. Not tested: games no model has seen. I designed three Corner Push, Mirror Twin, Charge Pad to remove the famous-game prior entirely, but stopped their training before the networks were ready. The SHA-256 results already show how much Astra's recognition relies on structure it can actually decode. Unseen games remain the cleanest test of the prior and are left for future work. Obfuscations that did not break it obfuscations-that-did-not-break-it These are the full results behind 3. Finding two: reading weights survived everything I threw at it, until runtime ran out 3-finding-two-reading-weights-survived-everything-i-threw-at-it-until-runtime-ran-out . Every transformation is stated in the prompt and leaves the network's function unchanged. Rescaling rescaling I reran the test through the same gateway with byte-identical prompts, 3 replies per network. I also built versions where the prompt truthfully says the input or output is multiplied or divided by 10, with the weights adjusted so the network computes exactly the same function. | Condition | Correct | Confidence | Reasoning tokens | |---|---|---|---| | Original 3 replies × 4 networks | 12/12 | 92–98% | 670–1,552 | | Input ×10 or ÷10 W1 adjusted | 16/16 | 94–98% | 470–1,976 | | Output ×10 or ÷10 W3, b3 adjusted | 16/16 | 95–98% | mean 1,146–1,220 | Graded obfuscation graded-obfuscation Next I stacked harder obfuscations, each stated truthfully in the prompt and each leaving the network's function unchanged. I graded them with the yardstick of Faith and Fate Dziri et al., NeurIPS 2023 : the size of the computation graph needed to undo them. | Condition | Depth / parameters to track | Accuracy | Mean confidence | Mean reasoning tokens | |---|---|---|---|---| | Original | 0 / 0 | 100% | 95.5 | 980 | | Per-channel scales | 1 / 5 | 100% | 97.1 | 457 | | Per-channel affine | 2 / 10 | 100% | 96.4 | 614 | | Grid flipped upside down | 1 / 1 | 100% | 96.4 | 832 | | Random permutation of all 500 inputs full table given | 1 / 500 | 100% | 92.9 | 1,507 | | Permutation + flip + scale | 3 | 100% | 92.5 | 1,917 | | + affine | 4 / 516 | 100% | 92.1 | 2,431 max 5,696 | | Givens rotations mixing input pairs, g = 10 / 50 / 250 | 2 / g | 100% | 96.5 / 95.5 / 92.5 | 1,238 / 1,507 / 3,706 | | Anonymous channels "A–E", no meanings | — | 100% | 86.6 min 78 | 2,059 | | Anonymous + flipped | — | 100% | 89.0 | 1,714 | | Hidden-unit shuffle + sign flips unstated control | 0 | 100% | 95.6 | 1,025 | Out of 156 runs, all 156 were correct. I couldn't fit a breaking point, because there was nothing to fit. What predicts the effort what-predicts-the-effort Spearman rank correlation of reasoning tokens with each complexity measure, over all 156 runs bootstrap 95% CI : | Measure | All runs | Arithmetic only | |---|---|---| | Description length bits | +0.33 +0.19, +0.49 | +0.45 +0.29, +0.58 | | Lookup-table entries | +0.29 +0.11, +0.44 | +0.46 +0.31, +0.60 | | Undo-graph size log₁₀\|V\| Faith and Fate | +0.10 −0.03, +0.27 | +0.12 −0.04, +0.31 | | Depth | +0.11 −0.03, +0.28 | +0.12 −0.01, +0.31 | Effort tracks how much there is to look up , not how much there is to compute . Faith and Fate's graph size counts the work of undoing all 16,000 weights, but Astra only undoes the rows it needs. Two exceptions: - Per-channel scaling made it think less than baseline. - Anonymous channels doubled the effort at almost no description cost. Inferring meaning costs more than arithmetic. The Table of Complexity the-table-of-complexity | Rule for the shuffle | Description | Computation log₂ ops | Accuracy | Confidence | Reasoning tokens | |---|---|---|---|---|---| | Affine: p = 337·j + 141 mod 500 | 64 bits | 10.6 | 8/8 | 94.6 | 1,348 | | Multiply-add then XOR-shift, ×1 | 64 | 11.6 | 8/8 | 92.2 | 2,425 | | … ×2 | 96 | 12.5 | 8/8 | 91.6 | 3,194 | | … ×4 | 160 | 13.4 | 8/8 | 89.9 | 2,982 | | … ×8 | 288 | 14.4 | 8/8 | 89.4 | 5,382 | | … ×16 | 544 | 15.3 | 6/8 | 77.9 | 7,001 | | … ×32 | 1,056 | 16.3 | 8/8 | 83.1 | 10,577 max 15,020 | | … ×64 | 2,080 | 17.3 | 6/8 | 64.4 | 12,464 max 18,126 | | … ×128 | 4,128 | 18.3 | 6/8 | 49.5 | 2,141 median 506; 7 of 8 replies stop early | | Order by SHA-256 of "wp-" + j | 48 | 20.1 | 6/8 | 43.8 | 399 | | SHA-256 on inputs and output columns | 64 | 20.2 | 2/8 | 15.6 | 467 | | breakout obfuscation.gif | What Astra sees as the shuffle gets harder. Left: the Breakout network as shown, each tile one input, brightness its largest first-layer weight. Right: what decoding a sample of rows recovers. Unshuffled, the weights are literally a picture of the board. | | | | | Growth rate of reasoning effort growth-rate-of-reasoning-effort How reasoning tokens grow with the work, fitted per reply. For the shuffles, x is the operations needed to decode one row the 76 runs where Astra decoded by hand, up to 64 rounds; 128 rounds is excluded because effort collapses there . For the arithmetic, x is Faith and Fate's computation-graph size |V|, using exact answers only. R² is on the original scale, except where marked log. | Task | Best fit | Exponent 95% CI | Power-law R² | Linear R² | Exponential R² | |---|---|---|---|---|---| | Decoding a stated shuffle | tokens ≈ 900 · ops^ 0.45 | 0.42–0.49 | log 0.87 | 0.76 | 0.54 | | n×n-digit multiplication | tokens ≈ 1.4 · \|V\|^ 0.98 | 0.93–1.03 | 0.75 log 0.98 | 0.90 | negative | | Best non-adjacent sum DP | tokens ≈ 3.7 · \|V\|^ 0.94 | 0.85–1.04 | 0.69 log 0.97 | 0.94 | negative | - All three are subexponential. An exponential is the worst fit in every case. For the arithmetic it's worse than a flat line, hence the negative R². - Arithmetic is linear: the exponents' intervals include 1, and the linear fit is the best one. Astra pays a roughly constant number of tokens per graph node, about 0.9 at 120×120 digits, because every node must actually be computed. - Decoding is about a square root, because Astra doesn't decode every row. As each row gets more expensive it samples fewer of them, so total effort grows much more slowly than the work per row. - Both stop growing at the limit. Beyond the breaking point, effort drops instead of rising: about 400 tokens for SHA-256, and 300–4,000 for declined multiplications and DP sums. Faith and Fate Probing faith-and-fate-probing For fun, I gave Astra the paper's own algorithmic tasks: zero-shot, no tools, exact match, one reply at high reasoning effort. GPT-4 collapsed by about 4–5 digits on multiplication. | Task | Sizes | Computation graph \|V\| | Correct | Reasoning tokens | |---|---|---|---|---| | n×n-digit multiplication | n = 2 … 120 | 16 → 29,280 | 32/33 one digit wrong at 80×80 | 17 → 27,183 | | Best sum with no two adjacent numbers | length 5 … 2,560 | 17 → 7,682 | 25/25 | 16 → 13,538 | - It's still exact at 120×120-digit multiplication. The only error was a single wrong digit in an 80×80 product 98.75% of digits right . - It uses a cheaper representation. Its reasoning summary reads "Calculating base-1000 product", then "Verifying multiplication result": it multiplies 3-digit chunks. - Effort tracks the paper's graph size. Reasoning tokens grow as |V|^0.98 for multiplication and |V|^0.94 for the DP problem. The chunking lowers the constant, to about 0.9 reasoning tokens per graph node at 120×120, but not the scaling. Unlike the weight-reading sweep, effort here follows Faith and Fate's |V|. - No hidden tools: every response contained only reasoning and message items. - It checks its arithmetic with checksums. In 12 multiplications from 20 to 120 digits re-run with reasoning summaries, 74 of 170 reasoning steps 44% were checks , and every reply checked at least once. The checks are modular: "Checking mod-9 residues", "Checking totals mod 11", "Checking M8 modulo 99999", the schoolbook trick of casting out nines, scaled up. 11 of the 12 were exact; the one error, at 100 digits, had the most checks of all 21 of 54 steps . Does the metric hold? Fitting the paper's compounding-error model, accuracy ≈ 1−ε ^|V|: - multiplication gives ε ≈ 7.6 × 10⁻⁶ per node, which predicts 50% accuracy at |V| ≈ 91,000 about 210×210 digits ; - the DP problem gives ε ≈ 4.9 × 10⁻⁵ per node, which predicts 50% accuracy at |V| ≈ 14,000 about 4,700 elements , matching the observed break at 5,120. So the metric doesn't hold across tasks. The two per-node error rates differ by about 6×, and the two tasks break at different |V|. Faith and Fate's graph size is not a task-independent measure of difficulty for this model. Where multiplication breaks: Astra stops answering. I doubled the size until it failed. At 240×240 digits |V| ≈ 116,000 it got 0 of 4 , but it gave no wrong numbers. Every reply declined. One reads: "I can't reliably compute this exact 480-digit product without tools, and I don't want to give you a fabricated number." Its reasoning summary ends with "Assessing computation feasibility", and it spent about 700–2,000 reasoning tokens, against about 27,000 at 120 digits. - The break lies between 120 and 140 digits. A parallel grid at 130–170 digits confirmed it: every valid reply from 140 digits up declined "I can't reliably compute this exact 280-digit product without tools…" , with no wrong numbers. That is earlier than the 210 digits the per-node error fit predicts: Astra stops answering before its errors would make it unreliable. - Effort collapses here too. As with SHA-256, Astra thinks less once it judges a task beyond its budget. - Scoring note: a reply without an ANSWER: line counts as wrong, so these abstentions count as failures. That's fair for "where does it stop being reliable?", but this break is abstention, not error. Where the DP sum breaks: errors first, then abstention. Unlike multiplication, the DP sum produces confident wrong answers before it starts declining: - 5,120 elements: 2 exact, 2 wrong numbers 18–22k reasoning tokens . - 10,240 elements: 2 wrong numbers 33–41k tokens, the most of any reply , 2 declines. - 20,480 elements: all 4 declined in about 300–500 tokens. Some declines hand back the program instead of the number: "I have not evaluated this on your sequence" , followed by the recurrence in Python. The likely difference from multiplication is input length: a 5,000-number sequence is long to read, and an off-by-one slip anywhere in it compounds silently, which Astra's checks don't catch. Compared with GPT-4 in the original paper compared-with-gpt-4-in-the-original-paper How far has the limit moved since 2023? I read GPT-4's zero-shot accuracies off Figure 2a of Faith and Fate and placed both models on the same scale: the size |V| of the computation graph from my own graph builder, used for both models , at the point where accuracy crosses 50%. | Task | GPT-4, zero-shot 2023 | GPT-4 at 50% | Astra, high reasoning 2026 | Astra at 50% | Astra / GPT-4 | |---|---|---|---|---|---| | n×n-digit multiplication | 99% at 2×2, 59% at 3×3, 4% at 4×4, 0% at 5×5 | about 3×3 digits, \|V\| ≈ 32 | exact to 120×120, declines from 140×140 | about 130×130 digits, \|V\| ≈ 34,000 | ≈ 1,050× in \|V\| 41× in digits | | DP over a sequence | 88%, 63%, 45%, 38%, 31% for lengths 2–6 | about 3.7 elements, \|V\| ≈ 13 | exact to 2,560 elements, 2 of 4 at 5,120 | 5,120 elements, \|V\| ≈ 15,000 | ≈ 1,170× in \|V\| about 1,400× in length | - Astra handles about a thousand times larger computation graphs than GPT-4 before dropping to 50%. - The factor is nearly the same for both tasks about 1,050× and 1,170× , even though their breaking points in |V| differ by about 2×. So |V| does not set one task-independent limit, but the jump from GPT-4 to Astra is roughly a constant multiple of it. - It isn't a like-for-like comparison of models alone. GPT-4 answered zero-shot in 2023, while Astra reasons for up to tens of thousands of tokens before answering, and that thinking budget is most of the difference. - The DP comparison flatters Astra. Faith and Fate's DP task asks for the chosen subsequence itself, while mine asks only for its maximum sum. The graph sizes come from my builder, not the paper's. Where the limit should be where-the-limit-should-be In Hutter's terms Universal Artificial Intelligence , the weight-reading results look like description length a stand-in for Kolmogorov complexity matters more than raw computation, because Astra decodes only what it needs. The arithmetic results show the other side: when every step must actually be executed, effort grows with the computation itself. But Kolmogorov complexity ignores running time. The runtime-aware measure is Levin's Kt = length + log time . Reasoning tokens buy time only linearly, and errors compound with every step. So I don't expect "just reason more" to be unbounded. Appendix B: Rebuilding the paper's unreleased games appendix-b-rebuilding-the-papers-unreleased-games The paper didn't release its four Experiment 1 games: grid versions of Breakout, Pong, Pac-Man and Freeway with a wall ring, 5 channels and 5 actions. I rebuilt them from the released networks and checked them against the paper's recorded numbers, using the released critic's TD residual as an independent check low on the true game . | Game | Verdict | Trained net, mine / paper | Random player, mine / paper | Critic TD residual | |---|---|---|---|---| | Breakout | validated | 21.96 / 21.71 | −0.42, 7.7 steps / −0.40, 7.8 steps | 0.0065 wrong rule sets: 0.08–1.55 | | Freeway | validated | 24.07 / 24.04 | 0.010 / 0.011 | 0.113 alternatives 0.136–0.271 | | Pong | approximate | 4.96 / 4.96 | −1.06 / −1.02 | 0.115; doesn't separate the candidates | | Pac-Man | approximate | 43.99 / 41.86 | 3.98 / 3.99 | 0.27; no ghost rule below about 0.25 | Breakout's rules came from asking the critic which next state it expects: - a 3-wide paddle; - the ball starts at 6,1 or 6,8 heading down and inward; - on hitting a brick or the paddle, the ball stays in its cell for a step and reverses; - a miss costs −1 and ends the episode. Even the sampled action rates match: .178 .184 .168 .234 .237 against the paper's .178 .183 .168 .237 .234 . This is not MinAtar Breakout: MinAtar's rules squeezed into the 8×8 area make the trained network lose the ball on the first drop. Appendix C: Experimental Setting appendix-c-experimental-setting See the source code: github.com/Flecart/weight-programming C.1 Experimental setup c1-experimental-setup Model and inference model-and-inference All experiments query gpt-6-astra through an OpenAI-compatible gateway Responses API at reasoning effort high , with no tools available. Each query is a fresh, single-turn conversation; sampling temperature cannot be set for this model. We record the reply text, the reasoning summary where requested, and token usage output tokens details.reasoning tokens . Requests that fail at the gateway are retried with backoff. Those that still fail are excluded from every analysis, never counted as wrong. C.1.1 Shuffled-network identification c11-shuffled-network-identification - Networks: The four PPO policies released with the original post Breakout, Pong, Pac-Man, Freeway : 500 → 32 → 32 → 5 tanh MLPs whose input is a 10 × 10 grid with 5 channels. - Prompt: The paper's which-game prompt, rebuilt byte-for-byte, with one change: a sentence stating the transformation applied to the network, and the transformed weights. - Transformation: Every transformation is a permutation of the 500 input rows of W1, so the shown network computes exactly the same function. Each one is verified to be a bijection, and the shown and original forward passes agree to within 2 × 10⁻¹² at full precision. Weights are printed at the paper's precision. Row j of the shown W1 holds original input p j , and p is given by a rule: - Affine: p j = 337·j + 141 mod 500. - XOR-shift, k rounds: start from x = j on the 9-bit domain 0..511 and apply, for r = 1..k, odd , then . Every round has its own constants , all listed in the prompt. If the result is ≥ 500, the k rounds are repeated cycle-walking until it falls in 0..499. We use k ∈ {1, 2, 4, 8, 16, 32, 64, 128}. - SHA-256: rows are ordered by the hexadecimal SHA-256 digest of the string "wp-" + j. In the output variant, the five logit columns are also reordered by the digest of "wp-act-" + a. - Replicates: Two independent replies per network and condition three for the untransformed control . - Scoring: The reply's Game: line is mapped to a game by keyword lists, and the earliest matching keyword wins. The reply is correct if the mapped game is the network's true game; "Undetermined" counts as wrong. Every decoded value the reply states in the form "p j = v" is parsed and compared with the true permutation. - Complexity measures: - Description length: 16 bits per stated constant plus 32 bits for the rule's wording. - Operations per row: the elementary operations needed to compute p j for one row. An affine map costs 3. An XOR-shift round costs 5, plus one comparison per cycle-walking pass 1.024 passes on average, measured . A SHA-256 hash costs about 2,300 word operations, and decoding one row requires hashing and sorting all 500 rows. - Total operations: the same, summed over all 500 rows. - Kt: description length + log₂ total operations . C.1.2 Faith-and-Fate arithmetic tasks c12-faith-and-fate-arithmetic-tasks - Multiplication: Two integers drawn uniformly from 10ⁿ⁻¹, 10ⁿ . Prompt: "What is a b? You have no tools: no calculator, no code. Give the exact product on the last line in the form 'ANSWER: