# I built a coding agent in ~970 lines of Python and benchmarked it honestly > Source: > Published: 2026-07-21 23:41:13+00:00 **TL;DR:** I built [nano-harness](https://github.com/TroyJLorents-GH/nano-harness) a coding agent in **~970 non-blank lines of Python** (5 files, 3 tools, 2 providers, MIT). It scored **59.6% (53/89) on the full Terminal-Bench 2.0 suite** with Claude Opus 4.8. Along the way an independent frontier model (GPT Sol 5.6) reviewed the code and gave it a **4/10**, which turned out to be the most useful thing that happened to the project. This is the whole arc the score, the bugs, and the parts that went wrong. Back in February I started reading about harnesses from none other than.... [Andrej Karpathy](https://github.com/karpathy/nanochat), I think is llm-wiki had just gone viral and I saw this repo. So I read it and started reading more on harnesses and loops. This was before EVERYONE was talking about loops and harnesses and every YT video was about prompts and so on. I wanted to learn something new, and at the time was training my own model using Azure Machine Learning (which costed an arm and a leg to learn), then was using PyTorch to learn patterns within my AI Operating System. Anyways, most agent projects I was following led with features: plugins, orchestration graphs, UI dashboards, memory systems (their obsidian "second brain") all the same thing. I had a harder time finding small, readable harnesses paired with a reproducible full-suite benchmark score (and if you know anything about wanting to benchmark your harness it **IS NOT CHEAP**!). This really stuck with me and I didnt know why more people were talking about this, to me this was where everything was heading and the harness is the part of an agent you can actually engineer, so it's the part you should be able to code and measure. So the thesis of nano-harness is **score-per-line-of-code**: the smallest readable harness that still puts up a real number on a real benchmark. Think of Karpathys nanochat, but for agent harnesses —small enough to read end-to-end in one sitting, honest enough that the number means something. My original goal was ~500 lines of code and to get a real coding benchmark above 45% and keep progressing.. This was going to be a challenge for me at least. The whole harness is one loop and three tools: `bash` `read_file` `edit_file` That's it. No embeddings, no planner, no sub-agents. `bash` subsumes ls, grep, build, test, and install. The model does the coding; the harness has exactly one job: **keep the run alive and keep it honest.** "Alive" means: retry transient API failures, truncate history before the context window dies, nudge a continuation when output gets cut mid-tool-call, kill and respawn the shell on a hang, and turn every exception into a structured result instead of a crash. "Honest" means: a failing command must *read* as a failure, and "done" must be earned. The load-bearing piece is a **verify gate**. Once a run has used tools, the first "done" is challenged: *re-read the task, run the relevant checks, prove it.* A later completion is accepted only when successful tool evidence has appeared since that challenge. If the pushback or iteration budget runs out without that evidence, the result is returned as `unverified` and therefor is not as success. Tool-free tasks (a pure question) are allowed to finish normally; there's nothing to verify. I benchmarked on **Terminal-Bench 2.0** via Harbor - 89 tasks, each in its own Docker container, with the exact shipping harness installed per container. My progression, in order: | Run | Config | Score | |---|---|---| | 10-task slice | Haiku 4.5, 50 iterations | 20% | | 10-task slice | Opus 4.8, matched budgets | 70% | | 10-task slice | Opus 4.8 + evidence-gated "done" | 80% | Full 89 | Opus 4.8, pre-hardening | 53.9% (48/89) | Full 89 | Opus 4.8, post-hardening | 59.6% (53/89) | The final run: 16.5 hours, two tasks in parallel, errored trials counted as failures (10 wall-clock timeouts on the heaviest tasks plus one container OOM all counted against me). Public Terminal-Bench results are agent-model *pairs*, so model quality and harness quality are entangled in every number. The official 2.0 board is topped by things like Codex CLI + GPT-5.5 (82.2%) and WOZCODE + Opus 4.7 (80.2%). I couldn't find a verified entry for my exact model on that table, so I'm not going to dress 59.6% up as a clean measurement of "the harness gap." It's my self-run result under the conditions disclosed here a ~970-line harness against a suite where the tuned, much larger harnesses live in the low 80s with the code and the task-level record in the repo for anyone who wants to check. Honestly I was proud of all the work I had put in over 3.5 months, while having my full time job as SR Engineer, and hosting a API with active paying users updating daily, and running a free web app with 100 active users and no marketing on anything besides a couple reddit post. Staying up till 2/3AM every night waking up at 5AM.. Which isn't healthy and I actually had a health scare due to lack of sleep and all the sitting I was doing despite working out everyday. But I think we can all admit that not only is learning now a days easier and much more fun, but with AI if you miss a day you feel like you were gone for a month and it can be overwhelming when there are weeks like these past few where its model after model, benchmark after benchmark. And on top of that I am working with what I got I dont have a mac-mini or NVIDIA SPARK, I bought the Beelink SER 10 MAX OpenClaw edition 64GB becuase it was the only 64GB available in my price range. And oh man do i wish I had a nice mini pc to run local models and test what I want. Back to the Harness Here's the part I actually want to tell you about. _And I just want everyone to understand that these benchmarks these frontier models score, the harnesses they have are thousands even tens of thousands of lines of code.. _ After the first full benchmark run, I pasted the five core files into an GPT Sol becuase it had just come out while I was 13 hours in to my first run. I used opus 4.8 to wrap my harness around so it was a "competitor's", no shared context and I asked for an adversarial/constructive code review. It scored the code **4/10** and produced a list of findings. The most consequential one was also the best catch: A shell command that failed was reported to the model as success. My bash tool captured output but not exit status. `false` , a failing test suite, `grep` with no match all of it came back looking clean and I felt like an idiot (...must of been the nights I woke up with head in the middle of my keychron.. also not happy about that). Which means my verify gate, the whole "keep it honest" centerpiece, could be satisfied by a *failing* test run. The harness's whole reason to exist had a hole in the middle of it. So I worked through the findings test-first, every fix landed with a failing regression test before the fix. (since this is an article about honesty: nano-harness was built with heavy AI-assisted coding. as im sure you figured by now, but unlike most I made sure i learned every step. I directed the work, made the calls, and ran every benchmark and review, but I did not hand-type all 967 lines, Id say again honestly I wrote ~100. The review gauntlet below used three independent review passes with no shared conversational context, which helped surface different classes of failures.) Then I had a second independent review pass over the result. It scored **6/10** and caught two of my fixes as only *half*-fixes: Fixed those too. Then a third pass a multi-agent cloud review fleet found exactly **two nits** (a missed retry case for client-side API timeouts, and an explicit JSON `null` argument bypassing a default). Fixed both. Total damage across the gauntlet: **~20 real bugs, test suite grown from 52 to 87 tests.** Among the fixes: `set -x` (a substring match used to latch onto the trace line and mis-frame every subsequent command)`edit_file` preserves file permissions (it used to silently strip the executable bit off any script it edited on Linux benchmark containers, that's a task-killer), edits `unverified` , not fake successAnd here's the _**punchline **_the benchmark handed me: the pre-hardening harness scored 53.9%. The post-hardening run same model, same suite — passed five more tasks for **59.6%**, a 5.7-point gain but also an extra ~180 lines of code. These are stochastic single runs, so I can't prove every point came from a specific fix. What I *can* say: the intervening changes were correctness and safety fixes, not task-specific benchmark patches, and the next full run scored higher. The biggest single fix was making failure *look like failure* to the model and it turns out models get further when their harness stops lying to them... who would have thought... More than I can quote to the exact dollar, and the honest reason is a gap in my own adapter: nano logs token usage to the agent's stdout instead of reporting it back into Harbor's result schema, so the run's summary JSON has null token fields. Adding up the surviving per-task agent logs gives about **3.0M tokens** (≈1.54M in, ≈1.43M out) and since a later retry pass overwrote a few of those logs, treat it as an estimate, not a receipt. At Opus 4.8's July 2026 list rates ($5/M input, $25/M output), that's roughly **$45** for the full 89-task run. A 10-task iteration slice runs one to two hours, on the order of $4–$6. Cheap enough that the bottleneck is wall-clock, not budget. The repo is [github.com/TroyJLorents-GH/nano-harness](https://github.com/TroyJLorents-GH/nano-harness) MIT, ~970 non-blank lines across `agent.py` , `tools.py` , `providers.py` , `prompts.py` , `cli.py` , with the 87-test suite, the Terminal-Bench adapter, the full benchmark breakdown, and the re-review prompt and resulting fix history in the tree. If you take one thing from this: **the harness's job is not to be smart the model is smart. The harness's job is to keep the run alive and refuse to let anyone lie, including the model, and including you.** The 4/10 review was the moment that thesis got real. Publishing the whole arc the bad score, the half-fixes, the variance is the point. *If you review the code and find bug #21, open an issue. That's the game.*