Harness Engineering 101: How Coding Agents Actually Work

Take one model and give it 169 real bug-fixing tasks from SWE-bench Verified. Keep the weights, the tasks and the context window exactly the same. Change only the agent system that runs around the model, and you will find bug-fixing task went from 43 to 72.

That result is from a paper that went up on arXiv in August, and it is the shortest answer I have to a question I get every week: which model should we pick? That question matters less every quarter. The frontier models sit close enough together that the software wrapped around them decides most of the outcome: what a task costs, whether the agent finishes it, and whether you can trust what it hands back.

That software is the harness. Designing it is what people have started calling harness engineering.

Birgitta Böckeler of Thoughtworks put it in four words, in an article on Martin Fowler's site: Agent = Model + Harness.

The model is the part you rent. Everything else is harness: the loop that keeps it working, the tools it can call, what goes into its context window, what it is allowed to do, and how its work gets checked before anyone accepts it. Claude Code is a harness. So is Codex CLI.

The field got here in three steps, and each one wrapped the step before it. Prompt engineering was about the words. Context engineering was about what else goes in front of the model with them: retrieved documents, memory, a summary of what happened ten turns ago. Harness engineering takes both and adds everything a model needs to act rather than talk.

The next ring is already forming. People have started calling it loop engineering: wrapping the harness in outer loops that re-run the agent on a schedule or an event, each run ending on a condition a machine can check, so nobody has to type the next instruction.

The easiest way to see a harness is to write one. This is the core of a coding agent in pseudocode. It is simplified, but every real harness I have read has this shape.

def run_agent(task, model, tools, limits):
    context = [system_prompt(), project_memory(), task]

    for step in range(limits.max_steps):
        if count_tokens(context) > limits.window * 0.8:
            context = compact(context)          # summarize old turns

        reply = model.generate(context, tools=tools.schemas())

        if reply.is_done:
            report = verify(reply)              # run tests, linters, a reviewer
            if report.passed:
                return reply
            context.append(report.as_feedback())
            continue

        for call in reply.tool_calls:
            if not policy.allows(call):
                result = ask_human(call) or "denied by policy"
            else:
                result = tools.run(call)        # most of the time: a shell command
            context.append(trim(result))        # keep the lines that matter

        if same_command_failed(context, times=3):
            context.append("That failed three times. Try a different approach.")

    return stop_and_report(context)

Count the lines that involve the model. There is one: model.generate. Every other line is a decision somebody had to make. When do you compact? How much of a 4,000-line test log does the model get to see? What does policy.allows say about git push --force? Change any of those answers and the same model behaves like a different agent.

Every long task eventually runs into the context limit, and what the harness does at that moment decides whether the agent finishes.

That is exactly what the August paper, "Same Model, Different Harness", changed. The new harness did two things. It shortened older tool results in stages as the window filled, and when it caught the agent repeating failed commands, it told it to try something else. Nothing else moved.

Give the model a 262K window and the gap nearly disappears. That is also why it matters in production, where every token is billed.

The usual techniques are compaction (summarize old turns), truncation (trim old tool output, keep recent output whole), memory files loaded at the start of every session (a project's CLAUDE.md or AGENTS.md), sub-agents that take a side task into a fresh context and return only the answer, and the newest one, a full context reset.

That last one exists for a reason you would not guess. Anthropic's team building long-running apps found that "compaction alone wasn't sufficient". As the window filled, models started showing what they call context anxiety: wrapping the work up early because they sensed the limit coming. A clean reset with a structured handoff file worked better than a summary the model knew it was running out of room behind.

An agent that can run commands can also delete things. Every harness picks a spot on a spectrum:

None of these is wrong. The right choice depends on how much a mistake can cost, which is a question about your machine and your data rather than about the tool.

This is the layer that lets you trust the agent without reading every line it writes. Böckeler splits it in two. Guides steer the agent before it acts: instructions, conventions, examples. Sensors check the result afterwards: tests, linters, type checkers, review agents. In the pseudocode, project_memory() is a guide and verify() is a sensor.

The catch is that an agent is a poor judge of its own work. Asked to evaluate what they produced, Anthropic found agents "tend to respond by confidently praising the work", even when a human can see it is mediocre. Their fix was three agents: a planner writes the spec, a generator builds, and a separate evaluator tests the running app with Playwright against criteria agreed before any code was written. The solo agent took 20 minutes and $9. The full harness took six hours and $200, and the result was far better.

Verification does not happen by itself. Someone has to build it, sometimes as a whole second agent whose only job is to be hard to please.

My first job was Linux server administration, and what hooked me was how much one line could do:

grep "Failed password" /var/log/auth.log | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head

Every IP that tried to brute-force SSH on the box, counted and ranked, from five programs that know nothing about each other. Watching a coding agent work gives me the same feeling. It reaches for the same kind of tools, in roughly the order I would:

$ rg -n "InvoiceTotal" src/
$ sed -n '118,160p' src/billing/invoice.ts
$ npm test -- invoice
$ git diff --stat

Give a model one tool, a shell, and it gets every program on the machine along with it. Nobody had to build a search_code tool or a run_tests tool. rg and npm test already existed, with decades of documentation behind them.

Four things make the shell fit a language model so well:

The vendors reached the same conclusion from the other side. Boris Cherny, who created Claude Code, has said early versions used RAG with a local vector database, and the team switched to plain agentic search (the model running grep and friends) because it worked better. Vercel cut an internal data agent down to little more than a single bash tool and reported it 3.5x faster on 37% fewer tokens, though on only five test queries.

The limits are worth knowing too. A shell cannot click through a web app. A SaaS product with no CLI comes in through an API or an MCP server. And the same shell that runs npm test can run rm -rf, which is why guardrails exist at all.

The harnesses I get asked about most, as of September 2026. Defaults change fast, so check the docs before relying on any cell.

Read the last column top to bottom. The harnesses that lean hardest on the shell ship the fewest tools, and Codex, the most shell-centric of the big three, is also the strictest about sandboxing it. That pairing is deliberate.

Whether you are picking one or building your own, measure the whole stack rather than the model alone:

Cost is the one people underestimate. Artificial Analysis's Coding Agent Index measured $0.07 to $2.26 per task across the model and harness pairs it tested. Most of that spread is the model, but the harness decides how many tokens the model burns on the way.

Anthropic has the best line on this: "Every component in a harness encodes an assumption about what the model can't do on its own." The same_command_failed check assumes the model will not notice it is going in circles. Compaction assumes it cannot hold a long task in one window. As models improve, some of those assumptions stop being true, and the parts built on them can go.

The part I don't expect to move is the permission boundary. A model can learn to catch its own mistakes. What it is allowed to delete on your production server stays your decision, written down in a harness, the same way it was in a sudoers file long before any of this.

The full version, including lifecycle hooks, agents beyond the code layer, and additional references, is available here: Harness Engineering 101