# OpenAI triples GPT-5.6 Sol's ARC score by preserving its memory > Source: > Published: 2026-07-30 00:04:06+00:00 OpenAI core products lead [Thibault "Tibo" Sottiaux (@thsottiaux)](https://x.com/thsottiaux) said GPT-5.6 Sol reached a state-of-the-art score on ARC-AGI-3 after OpenAI changed how the model's reasoning and context were carried between actions. The claim rests on a custom run of the benchmark's public tasks rather than ARC Prize's official verified leaderboard, where Anthropic's Claude Opus 5 remained the leader as of July 29th. In [a post on X](https://x.com/thsottiaux/status/2082609662231502932?s=46), Sottiaux wrote that the improvement required two setting changes: allowing GPT-5.6 Sol to preserve its private reasoning and letting it work across multiple context windows using compaction. OpenAI detailed the experiment in [a research post published July 29th](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/), reporting that the model's public-set score rose from 13.3% to 38.3% while consuming six times fewer output tokens. ([openai.com](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/)) Sottiaux joined OpenAI in 2024 after working on Google Maps and infrastructure for Google DeepMind systems including AlphaGo. He now oversees OpenAI's core products, including ChatGPT and Codex. That remit matters here because OpenAI says the revised benchmark harness reproduces context-management techniques already used in those products. The result offers a public demonstration of Sottiaux's product thesis: an agent's surrounding system can determine how much of the underlying model's capability reaches users. ([wired.com](https://www.wired.com/story/model-behavior-interview-with-openai-codex-lead-tibo-sottiaux/?utm_source=openai)) ### The harness changed the result ARC-AGI-3, released on March 25th, presents agents with unfamiliar, turn-based game environments without instructions, rules or stated goals. An agent must experiment, identify the objective and carry what it learns into later levels. ARC Prize designed the official harness to be generic, making comparisons across models easier and exposing weaknesses that model-specific tooling might conceal. ([arcprize.org](https://arcprize.org/blog/arc-agi-3-launch)) OpenAI said that generic setup created two problems for GPT-5.6 Sol. The official harness discarded the model's private reasoning after every game action, forcing GPT-5.6 Sol to reconstruct its understanding from a record of earlier moves and short notes. The harness also used rolling truncation: after the history exceeded 175,000 characters, it removed the oldest messages. OpenAI rebuilt the test with its [Responses API](https://developers.openai.com/blog/responses-api), passing the previous response ID so GPT-5.6 Sol could retain reasoning between actions. OpenAI then replaced rolling truncation with [context compaction](https://developers.openai.com/api/docs/guides/compaction), which condenses prior context instead of dropping the oldest actions outright. With those changes, OpenAI says GPT-5.6 Sol spent fewer tokens reconsidering each move, preserved discoveries over longer runs and formed more coherent strategies. OpenAI's implementation used a limit of 175,000 tokens rather than the official harness's 175,000-character threshold. The model scored 38.3% under that configuration, compared with 13.3% using the official setup on the same public task set. ([openai.com](https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/)) That gap is material for teams building production agents. Long-running software, browser and research agents repeatedly call tools, accumulate observations and approach context limits. Dropping an agent's reasoning or earlier actions can make a capable model repeat failed plans, lose track of intermediate findings and spend additional tokens reconstructing state. ### Claude still leads the verified leaderboard Sottiaux's "SoTA" description needs a narrower reading than the post suggests. OpenAI's 38.3% figure comes from its own harness on the public set. It has not replaced GPT-5.6 Sol's official verified result. [ARC Prize's GPT-5.6 results](https://arcprize.org/results/openai-gpt-5-6), published July 9th, show Sol at 13.33% on the public demo and 7.78% on the semi-private ARC-AGI-3 evaluation at maximum reasoning effort. ARC Prize described Sol as the first model to win a public ARC-AGI-3 game, while its performance across the semi-private evaluation remained in single digits. ([arcprize.org](https://arcprize.org/results/openai-gpt-5-6)) [Claude Opus 5's verified result](https://arcprize.org/results/anthropic-claude-opus-5), published July 24th, stands at 30.16% on ARC-AGI-3 at high reasoning effort. ARC Prize identifies Claude Opus 5 as the highest-performing verified model on the benchmark. ([arcprize.org](https://arcprize.org/results/anthropic-claude-opus-5)) The two scores therefore answer different questions. ARC Prize's leaderboard measures models under its standardized harness and semi-private tasks. OpenAI's experiment measures how GPT-5.6 Sol performs on public tasks when paired with OpenAI's production-oriented memory and context system. The 38.3% result demonstrates a large orchestration gain, while it does not establish an official leaderboard lead over Claude Opus 5. OpenAI's findings also expose a growing problem for model comparisons. A standardized harness improves comparability, yet it can suppress features that providers consider fundamental to their deployed systems. Model-specific harnesses can reveal production performance, while giving each vendor greater freedom to tune the test around its own architecture. For developers, the practical conclusion is more immediate. Choosing a model from a leaderboard captures only one layer of an agent stack. State retention, compaction, tool history and context limits can move performance enough to change which model appears strongest, along with how much it costs to run.