GPT-6 Astra’s Wild Score Swing Reframes the Race Toward AGI

OpenAI's GPT-6 Astra scored 62.7% on the ARC-AGI-3 benchmark under the Standard harness and 99.9% under the Provider Adapter harness, with identical weights and problems, according to ARC Prize. The Provider Adapter run cost $18,817 versus $26,098 for the Standard run and was about 3.66 times faster, highlighting how memory architecture, not just compute, is reshaping AI progress and chip roadmaps from Nvidia, Samsung, and SK Hynix.

A single AI model just posted two wildly different scores on the same test, and the gap between them is reshaping how the industry defines progress toward artificial general intelligence. OpenAI’s GPT-6 Astra scored 62.7% on the ARC- AGI https://www.kobaran.com/tag/AGI -3 benchmark under one testing setup and 99.9% under another, using identical model weights and identical problems. The only thing that changed was how much of its own prior work the model was allowed to remember. ARC Prize, the nonprofit that runs the benchmark, published the full breakdown this week. Under its provider-neutral “Standard harness,” which limits a model to a brief self-written summary between steps, Astra’s best run cost $26,098. Under a “Provider Adapter harness” that lets the model keep its own internal reasoning state and compress it as needed, the same model reached 99.9% for $18,817, roughly 3.66 times faster across comparable tasks. ARC Prize said the two setups answer different questions and are not directly interchangeable rankings. The result lands in the middle of a broader shift in how AI companies and chipmakers talk about progress. Where the industry used to frame capability gains around parameter counts and raw compute, engineers are increasingly pointing to memory architecture, meaning how a system stores, compresses, and retrieves what it has already worked out, as the harder problem left to solve. That shift is already showing up in chip roadmaps from Nvidia, Samsung, and SK Hynix, and it is likely to shape how future models are evaluated as much as how they are built. What ARC-AGI-3 actually measures ARC-AGI-3 is designed to test something multiple-choice benchmarks cannot: whether a system can drop into an unfamiliar, turn-based environment with no instructions and figure out the rules through exploration. ARC Prize built it specifically to track the distance between current AI systems and human-level general problem-solving, the working definition it uses for AGI. A model that beats the human baseline on efficiency is not the same as a model that has achieved general intelligence, and ARC Prize has been explicit that clearing this benchmark does not mean AGI has arrived. The two ways of scoring the same model The Standard harness gives every model provider the same minimal interface. After each move, the model has to decide for itself what to carry forward into a short written note, and everything else is discarded. It is meant to allow apples-to-apples comparisons across different AI companies. The Provider Adapter harness instead lets a model use its own maker’s native tools for managing memory: preserving its internal reasoning state across requests and compressing older context instead of deleting it. ARC Prize described this as measuring how a model performs once it can use the infrastructure its own provider built for it, rather than a stripped-down shared interface. Why the cost and speed also moved The memory-preserving setup was not just more accurate. It was cheaper and faster because a model that remembers its earlier reasoning does not have to redo that work from scratch on every step. ARC Prize reported that on the same set of problems solved under both harnesses, the Provider Adapter runs used about 49% fewer tokens in addition to finishing in roughly a third of the time. | Harness | Score on ARC-AGI-3 | Estimated cost | Relative speed | |---|---|---|---| | Standard harness | 62.7% | $26,098 | Baseline | | Provider Adapter harness | 99.9% | $18,817 | About 3.66x faster | Memory gaps are showing up across the industry, not just at OpenAI Astra is not the only case where the wrapper around a model, rather than the model’s own weights, decided the outcome. ARC Prize reported that Anthropic’s Claude Opus 5 scored about 30% on ARC-AGI-3 running on its own. When Nvidia built a separate agent system called AVO around the same model, one designed to manage memory, tool use, and error recovery across long tasks, Claude Opus 5’s score on the same benchmark’s public environments rose to a perfect 100. ARC Prize has said it now has repeated evidence that memory management, not model capability alone, is driving these swings. A model is only part of the system The pattern researchers are converging on is that a language model’s raw reasoning ability and the scaffolding wrapped around it, sometimes called the harness or agent system, are now separate variables that both matter. Nvidia’s own writeup of the AVO result put it plainly: the architecture around a model can unlock performance that the model alone cannot reach, independent of any retraining. Chipmakers are racing to build memory that keeps up Because so much now depends on how cheaply and quickly a system can store and retrieve what it has already learned, the memory hardware underneath these models has become a competitive battleground in its own right. Nvidia CEO Jensen Huang has said publicly that AI’s growing memory needs will reshape the storage industry and has repeatedly called the design of AI agents’ working memory one of the hardest open problems in AI infrastructure. Samsung, SK Hynix, and the new memory tiers South Korea’s chipmakers are positioning themselves around that demand. Samsung Electronics has unveiled a concept called zHBM, which stacks high-bandwidth memory vertically on top of an AI accelerator instead of placing it beside the chip, a design the company says could eventually deliver several times the bandwidth of its current HBM5 memory at a fraction of the power draw. Separately, SK Hynix and Sandisk have released an open specification for what they call High Bandwidth Flash, a NAND-based memory tier meant to sit between HBM and standard SSDs, aimed at giving AI inference systems far more capacity per dollar than HBM alone can offer. Compression could blunt the hardware boom Not everyone expects this shift to translate directly into unlimited demand for physical memory chips. Google Research has published a compression technique called TurboQuant that can shrink the memory a model needs to hold its own context by roughly sixfold with little to no loss in accuracy. If software-based compression keeps improving at that pace, some of the pressure that would otherwise flow to memory manufacturers could be absorbed before it ever reaches the hardware layer. What comes next These results are surfacing during one of the most crowded stretches of model releases the industry has seen, with OpenAI’s GPT-6 Astra arriving alongside new releases from Anthropic, Meta, and Google in recent weeks. As those companies converge on similarly capable base models, the harness and memory architecture wrapped around each one may end up mattering more to real-world performance than the next round of parameter increases. ARC Prize’s own conclusion is a narrower one than “AGI has arrived”: Astra’s harness-dependent scores show that memory management can be worth tens of percentage points on identical model weights, which is a significant finding about how AI systems should be built, even if it does not settle the larger question of when general intelligence will exist.