Your AI Coding Agent Doesn't Need a Better Prompt. It Needs a Harness.

A developer outlined a "harness engineering" approach for AI coding agents, arguing that reliable agent output depends less on prompt tuning than on the surrounding engineering environment. The walkthrough builds a lightweight Node.js harness — an AGENTS.md context file, a guardrails script, and a test suite run via npm run check — that gives an existing coding agent like Codex or Claude Code verifiable pass/fail feedback on its changes, with no paid API or npm dependencies.

Your AI coding agent just announced: ✅ Done Everything works. All tests passed. Excellent. Small problem: your project doesn't have any tests. So you write a better prompt. “Double check your work.” “Follow best practices.” “Act like a senior engineer.” You might as well add “pretty please” and offer it a cookie. The agent says “Absolutely ” and breaks something else. Here's my unpopular opinion: we spend too much time trying to write perfect prompts for agents working inside terrible engineering environments. A better prompt can help. But a prompt can't replace tests, project context, architectural boundaries, or an actual feedback loop. That's the idea behind harness engineering : design the environment around a coding agent so its work can be guided, checked, and corrected. And yes, we're building one. In about 15 minutes, with Node.js, no paid API, and no npm dependencies. The harness works with an existing coding agent that can read files and run terminal commands; we're not building another agent framework. What you'll build: AGENTS.md to explain the repository and its rules. npm run check that provides real feedback. A prompt says: “Don't break existing functionality.” A harness says: “Here are the regression tests. Run them.” A prompt says: “Stick to the architecture.” A harness says: “Read the architecture docs, and here's a check that rejects forbidden imports.” A prompt says: “Make sure it works.” A harness says: “Show me the exit code.” Not the same thing. Also, not enemies: a good prompt tells the agent what you want ; a harness gives it an environment in which progress is observable and verifiable . Task + constraints | v Coding agent <----- failed check output | ^ v | Code changes -- npm run check | pass / fail | Human review Let's make that real before the buzzword develops its own certification program. Prerequisites: Node.js 22+, Git, and a coding agent with access to your local repository Codex, Claude Code, or any equivalent . You can complete the entire exercise manually too. mkdir agent-harness-lab && cd agent-harness-lab npm init -y mkdir -p src test scripts docs .github/workflows Replace package.json with: { "name": "agent-harness-lab", "private": true, "type": "module", "engines": { "node": " =22" }, "scripts": { "test": "node --test", "guardrails": "node scripts/guardrails.js", "check": "npm run guardrails && npm test" } } Create src/pricing.js : js export function calculateTotal items, discountPercent = 0 { const subtotal = items.reduce sum, item = sum + item.price item.quantity, 0 ; return Math.round subtotal 1 - discountPercent / 100 100 / 100; } Looks fine, right? So does a chair with three legs, until someone sits down. This function accepts negative quantities, NaN prices, and 120% discounts. Let's make those failures impossible to ignore. Create test/pricing.test.js : python import test from 'node:test'; import assert from 'node:assert/strict'; import { calculateTotal } from '../src/pricing.js'; test 'calculates normal totals', = { assert.equal calculateTotal { price: 12.5, quantity: 2 } , 25 ; } ; test 'applies a valid discount', = { assert.equal calculateTotal { price: 100, quantity: 1 } , 15 , 85 ; } ; test 'rejects negative quantities', = { assert.throws = calculateTotal { price: 10, quantity: -1 } , RangeError ; } ; test 'rejects discounts above 100%', = { assert.throws = calculateTotal { price: 10, quantity: 1 } , 120 , RangeError ; } ; test 'rejects non-finite prices', = { assert.throws = calculateTotal { price: NaN, quantity: 1 } , TypeError ; } ; test 'does not mutate the cart', = { const items = { price: 10, quantity: 2 } ; const before = structuredClone items ; calculateTotal items ; assert.deepEqual items, before ; } ; Run: npm test Expected result: 3 tests pass, 3 fail. That's good news. We've converted a vague request into a measurable target. And yes, this example uses JavaScript floating point numbers to stay focused on agent workflows. For real money, use integer minor units or a decimal strategy with documented rounding rules. Create AGENTS.md at the repository root: Project instructions Context - Node.js 22+, ESM, no runtime dependencies. - Pricing logic: src/pricing.js - Tests: test/pricing.test.js - Architecture notes: docs/architecture.md Rules - Preserve calculateTotal items, discountPercent = 0 . - Do not change tests or guardrails to make a check pass. - Do not add packages without approval. - Do not access secrets, deploy, publish, or push changes. - Keep changes scoped to the task. Verification - Run: npm run check - Report the actual result, including failures. - Review git diff before claiming completion. Definition of done - Requested behavior works. - npm run check passes. - No unrelated changes. - Remaining risks are listed for human review. Create docs/architecture.md : Architecture The public API is calculateTotal items, discountPercent = 0 . Production code belongs in src/; tests belong in test/. The pricing function must be pure and must not perform I/O. Invalid inputs must throw, not silently produce nonsense totals. One subtle point: Codex recognizes AGENTS.md as a repository instruction file , but agent products differ. For other tools, verify their convention or explicitly tell the agent to read the file. A markdown filename isn't telepathy. And this file is guidance, not a security boundary . If an agent has permission to delete production resources, writing “please don't” in Markdown won't revoke that permission. Create scripts/guardrails.js : js import { readFileSync, readdirSync } from 'node:fs'; const pkg = JSON.parse readFileSync 'package.json', 'utf8' ; for const field of 'dependencies', 'optionalDependencies', 'peerDependencies' { if Object.keys pkg field ?? {} .length { throw new Error ${field} are not allowed in this lab ; } } for const file of readdirSync 'src' .filter f = f.endsWith '.js' { const source = readFileSync src/${file} , 'utf8' ; // Narrow teaching example: no imports in src/ .js. if /\b ?:import|require \s ?:\ | {\'" /m.test source { throw new Error Unexpected import in src/${file} ; } } console.log 'Architectural guardrails passed.' ; This checks two specific constraints: no declared runtime dependencies and no obvious imports in top level src/ .js files. It is not a general security scanner, and regex is not a robust JavaScript parser. For serious projects, use AST aware lint rules and dependency policies. Now run: npm run check The guardrail passes; the three behavior tests still fail. That's the harness doing its job, not an agent having a bad day. Open the repository with your coding agent and paste: Read AGENTS.md and docs/architecture.md before editing. Fix src/pricing.js so all existing pricing tests pass. Constraints: - Preserve calculateTotal items, discountPercent = 0 . - Negative quantities: RangeError. - Discounts outside 0..100: RangeError. - Non finite prices: TypeError. - Keep the function pure. - Change only src/pricing.js. - Do not install dependencies or weaken tests. Run npm run check and inspect its actual output. If it fails, fix the cause and rerun, up to 3 attempts. Do not claim success unless the command succeeds. Report files changed, check results, and remaining risks. Notice what changed: the prompt is still important, but it now points to an independent contract . A possible passing implementation is: export function calculateTotal items, discountPercent = 0 { if Array.isArray items { throw new TypeError 'items must be an array' ; } if Number.isFinite discountPercent || discountPercent < 0 || discountPercent 100 { throw new RangeError 'discountPercent must be between 0 and 100' ; } const subtotal = items.reduce sum, item = { if item || typeof item == 'object' || Number.isFinite item.price { throw new TypeError 'price must be finite' ; } if Number.isInteger item.quantity || item.quantity < 0 { throw new RangeError 'quantity must be nonnegative' ; } if item.price < 0 { throw new RangeError 'price must be nonnegative' ; } return sum + item.price item.quantity; }, 0 ; return Math.round subtotal 1 - discountPercent / 100 100 / 100; } Check it yourself: npm run check git diff --check git diff Expected after applying that fix: all 6 tests pass. Passing tests don't prove every possible behavior is correct, and git diff still needs human review. But at least we're discussing evidence instead of vibes. The useful loop is: php Edit - Run checks - Read real failure - Fix - Run again | Stop after 3 attempts and request review An agent that retries forever isn't autonomous. It's a subscription renewal strategy. Create .github/workflows/check.yml : name: Check on: push, pull request permissions: contents: read jobs: verify: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-node@v4 with: node-version: '22' - run: npm run check - run: git diff --check Open a pull request. Now CI runs the same checks as your local agent. For production use, pin third party GitHub Actions to vetted commit SHAs, enforce required checks and reviews, and protect tests and guardrails for example with CODEOWNERS and branch rules . An agent can make its own evaluation look green by weakening the evaluator if you let it edit everything. Also: run agent executed code in a suitably restricted environment. Tests don't replace sandboxing, permissions, or approval gates for destructive actions. Before, your quality strategy was: “Please be careful.” Now you have: | Problem | Harness response | |---|---| | Agent invents test results | Run npm run check and read the actual exit code | | Agent forgets project conventions | AGENTS.md + architecture notes | | Agent introduces forbidden imports | Architectural guardrail fails | | Agent breaks behavior | Regression tests fail | | Agent declares victory early | CI and human review remain separate gates | You can also preserve longer tasks using a short docs/HANDOFF.md with the objective, files changed, real verification results, blockers, and next action. Don't paste an entire chat transcript and call it documentation. Take five comparable tasks. Use the same model, starting repository state, allowed attempts, and time budget. Compare prompt-only versus prompt + harness . Track: Avoid pretending a single toy example proves a universal performance gain. The point of this lab is to demonstrate the mechanism, not manufacture a benchmark. Prompt engineering isn't dead. But “write a better prompt” is not a complete software engineering strategy. If you give an agent no tests, no context, no guardrails, and unlimited permissions, it can fail very efficiently. Congratulations: you've automated chaos. A harness makes failure visible and gives the agent a structured chance to recover. That doesn't guarantee correctness. It gives you a way to evaluate it. The goal isn't an agent that confidently announces: Everything is production ready 🚀 It's a workflow that can prove: ✓ Architecture checks passed ✓ 6 behavior tests passed ✓ Changes reviewed before merging → Ready for human approval What was the last thing your AI coding agent confidently broke? And more importantly: what single automated check would have caught it? Tell me in the comments. I collect these stories the way some people collect Pokémon. Further reading: