# AI Agents are a Context Management Strategy in a Trenchcoat > Source: > Published: 2026-08-26 16:00:00+00:00 My experience is that Claude Code is great with Foundation Models, but way too heavy for local inference. I’ve been enjoying oh-my-pi (omp) but was surprised that hax (a simpler agent in written in C) was TWICE as fast and used HALF the tokens! It’s all about PREFILL 1. Read on! ## Tl;dr When the model is a constant, the agent is a context-management strategy wearing a trenchcoat. For a fixed small sized coding task, a leaner agent with fewer tools will use less tokens and be faster (wall clock) to finish tasks. And it’s all about the [prefill](#fn:prefill). ## OK, the Longer Version I’ve been [saying for a while](https://blog.herlein.com/post/vibe-coding-review/) that agentic coding is a real sea change and not hype. But “it’s real” and “which one should you run” are two different questions. The second one deserves data, not vibes. ## The Setup: Change One Thing This is the mirror image of a project I did earlier where I held the *agent* constant and varied the *inference server*. This time I flipped it: one host, one model, one frozen task, one frozen test suite the agent never sees — and the only free variable is the agent itself. Everything underneath is nailed down. Same sglang server. Same model (`Qwen3.8-27B` weights, 262k context). Same sampling parameters. Same frozen task: implement `pngdec` , a Go CLI that parses PNG chunk structure and emits JSON. Same 32-case anchor suite that scores the result mechanically — no human reading diffs, no taste, no judgment calls. The agent self-terminates, the binary builds, it passes (or doesn’t) against tests it was never allowed to see. **Wall-clock time to that finished state is the primary metric. Number of tokens consumed is the secondary metric** Everything else is there to explain the time. An “agent” here means the whole bundle: its system prompt, its tool set, how it manages context, its turn loop, its stopping behavior. I do *not* try to normalize those away. That bundle is exactly the thing being measured. What I normalize is everything around it. The whole harness — task, scorer, adapters, raw per-trial artifacts — is public: [github.com/gherlein/compare-agents](https://github.com/gherlein/compare-agents). ## The First Bout: Heavy vs. Featherweight The opening matchup is deliberately lopsided in *design*, not in the model: **omp**(oh-my-pi) — a full-featured agent from the pi lineage. Rich tool set, model-role routing, skills, subagents, planning modes. The heavyweight. Everything you’d want if you believe more machinery makes a smarter agent.**hax**— a deliberately minimalist assistant written in C. Small tool set, small system prompt. The featherweight. Almost aggressively bare. Same model behind both. So raw token quality is identical *by construction*. If one wins, it’s not because it’s calling a smarter brain. It’s the same brain. The question is entirely about the harness wrapped around it. Here’s why the outcome wasn’t obvious to me going in. You can tell two opposite stories, and both sound right: - The richer agent finishes in fewer, smarter turns — better planning, fewer wasted edits. - The leaner agent finishes faster on the *clock*— smaller system prompt and leaner tool results mean less prefill per turn, and prefill cost compounds across dozens of turns on a local server. BOTH could be true. I genuinely didn’t know which effect would dominate. That’s the only kind of experiment worth running. ## The Result: hax Won, and Not By a Little On the frozen `pngdec` task, with the agent as the only free variable: **hax reached a finished solution 1.82x faster than omp** — median 45.7 minutes vs. 83.4 minutes. That’s not noise. The exact Mann-Whitney p is 0.00033, and the distributions barely touch: hax’s *slowest* trial was 61.7 minutes, omp’s *fastest* was 59.2. Every other omp run was slower than every hax run. That’s about as clean a separation as you ever get from stochastic agentic runs. And correctness? A tie. Both agents finished 100% of their trials with a median anchor score of 32/32. Now, here’s where the engineering instinct kicks in and I have to be honest with you: **that tie is a ceiling effect, not proof they’re equally good.** The anchor suite maxed out. 14 of 17 binaries scored a perfect 32/32, the other three scored 31/32. When your instrument is saturated, it isn’t measuring anything anymore. If you need to separate agents on *correctness*, `pngdec` is too easy and the spec needs harder cases. I’m not going to pretend a pegged needle is a result. ## Why hax Won: It’s the Baggage, Not the Brains This is the part I find genuinely interesting. The time difference is **entirely about context volume, not generation.** Same model, so the model does the same amount of work per call. And the data confirms exactly that — generation tokens per request are statistically *indistinguishable* between the two agents (p=0.81). The brain works identically. That’s not a disappointment; it’s the control working. It tells you the measurement is sound. Where they diverge is prefill: - omp sends **2.38x more prompt tokens per request**(p=0.00016 — the strongest effect in the whole dataset). - omp makes **1.80x as many requests**(median 34 turns vs. 20). - omp spent longer just *orienting*— 4 to 6 turns before its first edit, vs. 1 to 2 for hax. - omp triggered context compaction in 4 of 9 trials. hax never compacted. Not once. Here’s the decomposition in one table. Note the *per request* row — that’s the one that separates “smaller prefill” from “fewer turns”: | metric | omp | hax | ratio | |---|---|---|---| prompt tokens per request | 34,235 | 14,414 | 2.38x | | server requests (turns) | 36 | 20 | 1.80x | | total prompt tokens | 1,095,525 | 288,696 | 3.79x | It’s *both*, and they compound. Even if you forced both agents to take the same number of turns, omp would still send 2.38x more tokens on every single call — that’s the agent bundle itself (bigger system prompt, more tool definitions, more verbose tool results resent each turn), not turn count. Then, separately, omp takes more turns on top of that. Multiply it out: 1.80x the requests times 2.38x the prefill per request lands right on the ~3.8x total prompt-token ratio. The decomposition is consistent. That prefill is the *agent bundle* — the bigger system prompt, the more verbose tool results, the larger conversation resent every single turn. “But prefix caching!” you say. I checked. Both agents ran high cache-hit rates (omp 92–98%, hax 90–95%), so most of omp’s larger context *was* cached — and it was still 1.82x slower on the clock. The volume itself is the cost. Caching softens it; it doesn’t erase it. So the mechanism is clear: on this task, the heavier agent’s machinery didn’t buy better code. It just cost more time carrying itself around, turn after turn, on a local inference server where prefill compounds. ## What This Does NOT Prove I want to be careful here, because it would be easy to over-read this into “minimalism always wins” and that is *not* what I’ve got. `pngdec` is a fairly contained task. The anchor suite saturated. I did not measure whether omp’s planning and subagents pay off on something genuinely hard — a big refactor, a sprawling codebase, a task with real architectural forks in it. That’s exactly the kind of terrain where more machinery is *supposed* to earn its keep, and I haven’t tested it. On an easy task, all that orientation overhead is pure tax. On a hard one? Maybe it’s the whole ballgame. Dunno yet. That’s honest. There are other limits I’m not going to bury: **n is a hair below target.** I wanted at least 10 trials per agent; this run had 9 and 8. The time effect is miles past significance, but the completion-rate comparison is uninformative because neither agent ever failed.**The anti-gaming cross-matrix is void.** Every agent wrote its own test suite, and all 17 of them disagreed with the reference implementation — because`REQUIREMENTS.md` doesn’t pin down error-recovery semantics, and each agent invented its own answer. That’s a spec bug, and fixing it is a prerequisite for that metric to mean anything next time.**Single machine, single ~19-hour batch.** Interleaving the runs controls for drift*within*the batch. It can’t rule out a machine-specific effect. The full write-up doesn’t hide any of this. It’s all in the findings, with the per-trial data and the harness bugs I found while scoring: [FINDINGS-omp-vs-hax.md](https://github.com/gherlein/compare-agents/blob/main/docs/FINDINGS-omp-vs-hax.md). ## What’s Next: Bring in the Go Minimalists The obvious next move is to widen the field, and I’m taking it toward the language I keep [arguing is right for this era](https://blog.herlein.com/post/why-go-for-robotics/). The next round adds **pi** and **kit** — two more minimalist agents, both written in Go. I actually tried to run them in this batch. They aborted silently on 6 of 10 trials — and it wasn’t an agent-capability problem, it was a harness problem. The stall-tolerance timeout was pinned for omp and hax but not for kit and pi, so when the server hiccuped, omp retried and kept going while kit and pi just exited. So the plan is: fix the harness, then run omp, hax, pi, and kit against the same frozen yardstick. Heavy vs. lean vs. lean-in-Go. Same brain behind all of them. I want to know if the featherweight result holds when the featherweights are written in a language built for exactly this kind of single-binary, no-drama deployment. ## Look The headline is fun — the tiny C agent smoked the feature-rich one by nearly 2x — but the *reason* is the real takeaway. Every token your harness carries per turn, you pay for, over and over, across dozens of turns. On a local server, that bill is the whole game. That doesn’t mean throw away planning and subagents. It means: measure your harness like it’s part of your infrastructure, because it is. And hold something constant before you go arguing about which agent is best. More runs coming. If this was useful, or if you think I’m wrong about something — and on the “does heavy pay off on hard tasks” question I might well be — drop me a note on [LinkedIn](https://www.linkedin.com/in/gherlein/). And yes: pay it forward. - **Prefill** is the phase where the model reads the entire prompt — system prompt, tool definitions, and the whole conversation so far — and computes its internal state before generating a single new token. It’s distinct from*decode*, the phase that actually produces the output tokens one at a time. Prefill cost scales with how many tokens you feed in, and an agent resends most of its growing context on*every*turn, so a heavier agent pays that prefill tax over and over across dozens of turns. Prefix caching can reuse the computation for an unchanged leading chunk of the prompt, but it softens the cost rather than eliminating it.[↩︎](#fnref:1)