What Is an AI Agent Harness? Why Does It Matter for the Model? OpenAI's GPT-6.2 Astra scored 70.3% on ARC-AGI-3 with ARC Prize's Standard harness and 99.8% with its Provider Adapter harness at the same max reasoning effort, per ARC Prize leaderboard data as of November 17, 2026, at a cost of $4,107 versus $24,169. A May 2026 position paper, "Stop Comparing LLM Agents Without Disclosing the Harness," found that swapping harnesses for one model moved pass@1 on 100 SWE-bench Verified tasks by 8.5 to 13.0 points, while swapping models under one harness moved scores by 2.5 to 5.0 points, with two models trading places in 6 of 9 cross-harness comparisons. The author concludes an agent score belongs to the model, harness, settings and test setup together, and that a score without those details cannot be compared with another. Hi everyone. OpenAI’s GPT-6.2 Astra scored 70.3% on ARC-AGI-3 with one harness and 99.8% with another. Same model, same benchmark, same reasoning effort max . ARC Prize’s leaderboard lists both numbers, data as of November 17, 2026. If you didn’t know, a harness is the code around a model, the loop that calls it, the tools, the memory, the checks. In March 2026 LangChain’s Vivek Trivedy wrote it as a formula, “Agent = Model + Harness”. Since then a popular claim says the harness matters more than the model. This story was written with the assistance of AI: Claude helped draft the text, and ChatGPT made the cover and the diagram. The short version: You know that a model on its own only writes text. To work as an agent, it needs code that calls it in a loop and runs the tools it asks for. The same code decides what the model sees, keeps memory between steps, limits the damage it can do and checks its work. That code is the harness . For example, Claude Code and Codex are harnesses around a model. The name “harness engineering” took off in February 2026, when Mitchell Hashimoto used it. An eval harness is something else, it is just the code that runs a benchmark. The same model can score very differently. ARC Prize ran GPT-6.2 Astra twice with the same settings. The first run used a basic harness ARC calls it Standard . The second used a harness that lets the model remember what it thought in earlier steps ARC calls it Provider Adapter . The second run scored much higher and cost far less: $4,107 against $24,169. It is not a one-off. At max effort, Claude Fable 5.2 goes from 65.7% to 94.9% and GPT-6.2 Sol from 54.9% to 98.2% when ARC Prize swaps the harness. The gain depends on the model: Claude Haiku 5.5 only goes from 0.5% to 2.8% . Two more things helped. In February 2026 OpenAI described a product of about a million lines of code, built in about 1,500 pull requests, with no code typed by hand. And a team can change the harness, but not the model it rents. Most numbers here come from the companies that built the tools, so treat them as signs, not proof. The most controlled test I found is a May 2026 position paper, “Stop Comparing LLM Agents Without Disclosing the Harness”. The authors ran 3 models GLM-5.1, GPT-5.4, Kimi K2.6 with 3 harnesses minimal, improved, full . Each pair ran the same 100 SWE-bench Verified tasks, twice, and was scored by pass@1 the share of tasks solved on the first try . Changing the harness for one model moved the score by 8.5 to 13.0 points. Changing the model with one harness moved it by 2.5 to 5.0 points. In 6 of 9 comparisons across harnesses, two models swapped places. But the paper has limits. The authors built the three harnesses themselves, the sample is small, and they only claim this for long multi-step tasks and models of similar strength. Other work points the other way: Harness effects are real and can be large. They depend on the model, the task and the setup. The harness is real engineering, and it produced the biggest gaps above. The numbers do not prove it matters more than the model. They do show that an agent score belongs to the model, harness, settings and test setup together . A score without those details cannot be compared with another one. Which part of your own agent setup would you measure first? If you enjoyed this article, I’d be grateful for your support I’m a Product Manager with an engineering background ex SWE , focused on building, scaling and growing products. I’m especially interested in new technologies, particularly AI/ML, and how they can be applied in real workflows. Email: akzhankalimatov@gmail.com mailto:akzhankalimatov@gmail.com ✅ Connect with me on LinkedIn https://www.linkedin.com/in/akzhankalimatov/ , X Twitter https://x.com/akzh4n ✅ Press and hold the 👏 to give up to 50 claps for this article What Is an AI Agent Harness? Why Does It Matter for the Model? https://pub.towardsai.net/what-is-an-ai-agent-harness-why-does-it-matter-for-the-model-73dca33707a1 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.