The middle path between a stock agent and a custom harness

A developer proposes a middle path between stock coding agents and custom harnesses, introducing pi, an open-source terminal coding agent that ships a general terminal-agent harness with replaceable coding-specific components. The developer tested pi against five real workflows, arguing it offers a cost-effective alternative that avoids the pitfalls of both extremes.

Here’s a conversation I keep watching play out. An engineering lead wants their team on a coding agent. Compliance won’t sign off on the stock one — no audit log, no way to block writes to secrets, no way to make destructive shell commands ask twice. So the options on the table narrow to two: keep waving people off the tool they want, or build a custom agent from scratch on top of an SDK. Two engineers, a quarter, and now the team owns a product nobody wanted to own. The conversation is the same at every company I’ve seen it in. It’s the wrong choice on both ends. Stock coding agents are close enough that the gap feels small. A custom harness is far enough that you don’t want to cross the distance. There’s a middle path most teams haven’t noticed, and I want to make the case for it. The middle path is pi https://pi.dev , an OSS terminal coding agent that ships the boring parts of the harness and hands you the interesting parts as first-class extension points. Custom tools, tool-call gates, commands, skills, prompt templates, the system prompt itself. All of it lives in your repo as normal TypeScript and Markdown. No fork. No SDK-level plumbing. If you delete .pi/, you’re back to a stock coding agent. I spent a weekend testing this claim against five real workflows. Three coding cases, two non-coding cases. The non-coding ones matter, and I’ll get to why. Stock agents are the fastest thing to adopt. They also have a small surface. You get some combination of a system prompt override, MCP servers, and a permission list. That’s usually it. When you hit a real workflow constraint an auditor who wants a log, a platform team with an opinionated deploy pipeline, a research group that wants a locked-down read-only reviewer the shape of the tool doesn’t bend. The other end is building your own. Every SDK sample makes it look easy. The tool loop is fifty lines. And then you need streaming. And cancellation. And session storage that doesn’t corrupt on Ctrl-C. And a TUI that doesn’t jitter. And compaction that doesn’t drop the last user message. And a model router that handles four providers. And permission gates. And a way to update the model catalog without shipping a release. Six months later you have a product, and the product is worse than the OSS agent you started with, and the two people who understand it are the only two people who can touch it. The cost curve is below. Stock red is cheap up front and expensive the day a real constraint lands. Custom black is expensive up front and expensive forever. There ought to be a floor between them. Harness assembly green is that hypothesized floor. Here’s the distinction the whole hypothesis rests on. Pi is nominally a coding agent. But only two things about it are coding-specific: 1. The four built-in tools: read, write, edit, bash. 2. The default system prompt. Both are replaceable. The built-ins go away with --no-builtin-tools. The system prompt is a flag or a file. Everything else pi ships is not coding-specific. It’s just what any competent terminal agent needs. That’s the observation. If those two coding-specific pieces are replaceable, then pi isn’t really a coding harness. It’s a general terminal-agent harness that happens to ship with a coding-agent profile enabled by default . Every “custom harness” a team actually wants is really some combination of six moves. Pi exposes all six as normal repo files. The cost of each roughly goes A < B ≈ C < D ≈ E < F . A team starts at A and adds strategies as pain shows up. None of them require touching pi’s internals. I set up a repo, pi-harness-assembly https://github.com/tacoda/pi-harness-assembly , with five cases. Three coding, two non-coding. The point wasn’t to build production tools; it was to see whether the taxonomy holds up when you point it at real needs. Case 1: a regulated shop. Every tool call logged to disk, writes to .env and prod-config paths hard-blocked, a confirmation before destructive shell commands run. This is strategy D plus A. One extension file, about 130 lines including comments. It reacts to tool call events, appends a JSONL audit line, checks the path against a protected-paths list, and prompts the user for dangerous bash patterns. The auditor gets their log. The team keeps the agent they wanted. If someone hates it, they delete one file. Case 2: an internal platform team. They want a canonical deploy workflow preflight, canary, verify, promote that the agent can execute but can’t skip stages of. This is strategies E, C, and B together. An extension registers a deploy service tool that shells out to the real deploy pipeline. A /deploy command kicks off the flow. A skill under .pi/skills/deploy-runbook/SKILL.md contains the actual playbook: run the stages in order, stop before prod promote, don’t batch services. The playbook lives in Markdown so the platform lead can edit it without touching TypeScript, which matters more than any technical detail on this list. Case 3: a research group. They want a locked-down “explain and critique my notebook” mode — read, grep, list, nothing else. Strategies F, C, and B. A bin/pi-review launcher runs pi --tools read,grep,find,ls, expands a prompt template, and preloads a skill with a review rubric correctness, reproducibility, statistical soundness, performance . No writes possible. The model describes changes instead of making them. Twenty lines of shell and two Markdown files. Those three are the interior of the hypothesis. Coding workflows, coding tools, coding-shaped constraints. If pi is a middle path, it should handle them, and it does. The two non-coding cases are where the hypothesis either generalizes or breaks. Case 4: a web scraping harness. Not a coding agent. read, write, edit, bash are all gone. The extension registers fetch url and save record. A /scrape command starts a job. A skill contains the extraction playbook: probe with a headers request, fetch text-mode HTML, identify the repeating entity, save one JSONL record per entity, paginate carefully. The launcher passes --system-prompt “$ cat .pi/systems/scrape.md ” to reframe the agent’s identity for that invocation only, so it doesn’t leak into the coding cases in the same repo. Case 5: a research assistant. Also not a coding agent, and structurally different from scraping. Where scraping extracts rows from known URLs, research finds sources, reads them, and synthesizes with citations. Four tools: web search, fetch source, add note, list notes. Every claim gets a citation to a URL the agent actually fetched because the skill mandates it, the extension can’t enforce it, and that’s fine because the skill is where the rule belongs. Notes accumulate as Markdown under research-notes/<topic .md across sessions, and list notes gives the agent a way to discover its own prior work. Long-term memory built out of the filesystem. The three coding cases were what I expected. Small extensions, plain Markdown, working workflows. Not novel; this is what pi is for. The two non-coding cases are what changed my mind about the scope of the claim. Case 4 and Case 5 use the same six strategies as Cases 1–3. Same taxonomy, same file layout under .pi/, same shape of extension. The only new move is passing --system-prompt “$ cat … ” in the launcher instead of relying on the default. That’s it. Two unrelated domains — scraping and research — produced structurally identical harnesses. That symmetry is the finding. If the six strategies applied to coding but not to anything else, the hypothesis would be “pi is a good coding harness with room to customize,” which is a much smaller claim. What I actually saw is closer to: pi is a general terminal-agent harness with a coding profile as its default, and repointing it at a different domain costs one extra file the replacement system prompt . The other thing worth naming: the same person can maintain all five cases. The extension files are 60 to 200 lines each. The skills are Markdown a domain expert can edit. There’s no shared framework code because pi is the framework. The team doesn’t own a harness product; they own a small pile of configuration. I want to be careful here because I’ve seen “just use OSS X and extend it” arguments fall apart in production. A few honest caveats. The extension API is TypeScript. If your team doesn’t write TypeScript, every strategy above D costs more than I’m making it look. You’d still get A, B, and C for free, and that’s not nothing. The deploy runbook and the review rubric are Markdown, but the enforcement layer needs code . Extensions run with full system access. A malicious pi package can do anything your shell can do. This is the same threat model as installing an npm dependency, and the same mitigation applies: read the code before installing anything from a stranger. Pi’s docs are direct about this. Teams with tight supply-chain controls need to account for it. Skills are guidance to the model, not enforcement on the model. The scraping skill says “don’t spider off-domain unless the user asked.” A determined model can still do it if the tools allow it. If a constraint has to be true, put it in a tool gate strategy D , not in a skill. And the whole argument rests on pi staying maintained. It’s an OSS project. The “expensive forever” costs of the build-it-yourself path come back if the project you’re depending on goes cold, you inherit the maintenance you were trying to dodge. That’s not a reason to avoid the middle path, but it’s a risk worthy of consideration. If you’re on a team that wants a coding agent and can’t ship one because of a real constraint, here’s the smallest version of the experiment. Pick your single sharpest pain point. Not five. One. If it’s compliance, that’s a tool-gate extension : sixty lines. If it’s a specific workflow you keep hand-holding the agent through, that’s a skill : a Markdown file. If it’s “I need a locked-down mode for the intern,” that’s a launcher script and a --tools flag. Write only that. Ship it to one person. See if it holds up for a week. If it does, add the next thing. If it doesn’t, delete it and you’re back to stock pi with no sunk cost. The reason to bother with the middle path isn’t that it’s technically clever. It’s that it lets you match the agent to the actual constraint without buying a product you don’t want to own. Every strategy in the taxonomy is reversible. You can start at A on a Tuesday and be back at stock pi by Friday if it wasn’t worth it. That property — cheap to try, cheap to unwind — is the whole point. The reason I care about this is that I keep meeting teams who’ve decided the choice is binary. It isn’t. The audit-and-guard extension in the test repo is 135 lines. The deploy workflow is 81. The scraping tools are 165. None of them are impressive engineering. That’s the point. The interesting work happens above them, in the skills and the workflows and the constraints that actually match the team. The taxonomy exists to keep the code layer small enough that the team can afford to focus on the layer that matters.