cd /news/artificial-intelligence/tooling-every-ai-software-harness-sh… · home topics artificial-intelligence article
[ARTICLE · art-107180] src=dev.to ↗ pub= topic=artificial-intelligence verified=true sentiment=· neutral

Tooling every AI software harness should have

A developer argues that AI coding agents require enforced tooling, not just advisory documentation, to maintain code quality. The post recommends 100% branch coverage on touched files, mutation testing, type checking, and linters that agents can run and fix before human review. The author notes that the economics of enforcement have shifted as agent time is cheaper than human time, making previously uneconomic thresholds practical.

read8 min views2 publishedAug 22, 2026

When an agent writes the code, you stop reading every line. You can pretend otherwise for a while, but on any real codebase you end up skimming a 600 line diff and approving it because nothing jumped out. Nothing jumping out is not the same as nothing being wrong.

Documentation and a well tended CLAUDE.md only take you so far. The longer an agent runs, the more those instructions get crowded out of context and quietly forgotten. Instructions are advisory. Tooling is not.

Two ideas run through everything below.

The first is that the economics of enforcement have changed. Every quality threshold we set as an industry was really a judgement about how much human time it was worth spending. 100% branch coverage was not wrong, it was expensive, so we called it diminishing returns and settled at 80%. That cost is now paid in agent time, which is cheap and which you are not spending your evening on. Thresholds that were uneconomic five years ago are now just settings. Turn them up.

The second is that a tool only counts if the agent sees the output. A linter that fails in CI thirty minutes after the agent stopped is a message to you. A linter the agent runs itself, reads, and fixes before you ever see the branch is part of the harness. Same tool, completely different value. Everything on this list should be reachable by the agent with one command, and should fail loudly enough that it cannot be ignored.

The obvious one, but push it further than you would for a human team. Have a testing framework, and set minimum line and branch coverage to 100%.

That number is doing something specific. At 80%, an agent will happily leave the awkward branch untested, because the awkward branch is exactly the one that is annoying to set up. At 100% there is no judgement call left to get wrong, and no negotiation to have.

On an older codebase this is not feasible in one go. What does work is applying the 100% minimum only to files touched in a PR. Coverage ratchets up as the agent works through the code, and you get the high standard on new work immediately.

I use: https://rspec.info/ and https://github.com/simplecov-ruby/simplecov Agents love writing code. Left alone they add checks that look defensive and do nothing. An empty check before looping over an array is the classic: looping over an empty array already does nothing, at the same speed, so the check is pure noise that a reviewer now has to read and reason about.

Coverage will not catch this. The line is executed by your tests, so it counts as covered. Mutation testing will, because it changes that line and finds no test cares. It is the only tool I know that answers both halves of the question at once: is every concept actually covered by a test, and does every piece of code actually add value.

Have a look at [https://stryker-mutator.io/docs/](https://stryker-mutator.io/docs/) for the basics.

I use: [https://github.com/mbj/mutant](https://github.com/mbj/mutant)

A quick win, and useful to agents in a way it is not to humans. An agent generating code from a half remembered API will produce a method call that looks entirely plausible and does not exist. Type checking closes that gap statically, before anything runs, and gives the agent a precise error to act on rather than a stack trace it has to interpret.

Even dynamic languages have a route in now. TypeScript for JavaScript, Sorbet for Ruby.

I use: https://sorbet.org/ Every mature language has tooling that flags patterns that are outdated or actively harmful. This matters more with agents than it did with people, because an agent's instinct comes from its training data, and its training data is full of code written years ago. It will reach for the idiom that was correct in 2019 with total confidence.

Rubocop catches exactly this. In a Rails codebase it will flag Time.now

where you want Time.current

, the timezone safe equivalent. That is not a style preference, it is a bug, and it is the sort of bug an agent will reintroduce every few weeks unless something stops it.

There are linters for CSS, HTML, Go, JavaScript and most other things. If the language is mature, one exists.

I use: https://github.com/rubocop/rubocop, https://github.com/Shopify/erb_lint, https://eslint.org/, https://github.com/remarkjs/remark-validate-links, https://github.com/hadolint/hadolint

Agents write insecure code. The saving grace is that they are insecure in well documented ways, because they learned it from existing codebases, and those mistakes already have detectors built for them. String interpolation into a query, a secret pasted into a config file, an over permissive workflow permission: all findable automatically.

Non negotiable on any codebase where agents write a meaningful share of the code.

I use: https://brakemanscanner.org/, https://github.com/gitleaks/gitleaks, https://github.com/zizmorcore/zizmor/

Agents take copy paste to the next level, and this is one of the few problems that gets harder specifically because you did not write the code yourself. When you duplicate a block, you know you did it, and you feel the itch. When an agent does it in a file you have not opened, there is nothing to notice. The diff looks like new code, because to you it is.

A detector gives you the thing your memory would have given you if you had been typing.

I use: https://jscpd.dev/ and https://github.com/seattlerb/flay This is where the list turns into a harness.

Every tool above is worth ten times more when the agent runs it, sees the failure, and fixes it, versus when it fails in CI and waits for you. Hooks are the cheapest way to force that, because they sit at points the agent cannot route around.

The part worth taking advantage of is that there is more than one such point. An agent's work has natural stages, editing a file, committing, pushing, and you can attach different checks to each. Feedback is cheapest at the earliest stage where the check can meaningfully run, because the further the agent gets from the code it wrote, the more context it has to rebuild to fix it. So push each check as early as it will go, and let cost decide how much further it slides.

Post edit is the tightest loop available. If your type checker is fast, run it here rather than waiting for the commit: the agent gets the error while the file it just wrote is still the thing it is thinking about, which is a much cheaper fix than discovering it three files later. Same for linters, if they are fast or can be pointed at a single file. Immediate feedback on one file beats a list of twenty violations at the end.

Pre-commit is for anything that needs the whole change rather than one file, or is too slow to run on every edit. The full lint pass, the formatter, secret scanning, the fast part of the test suite.

Pre-push is the last cheap stop before CI. Full tests, duplication detection, the slower security scanners. Getting a failure here still costs the agent a minute. Getting it from CI costs a round trip and usually costs you as well.

Set up right, the agent commits, the hook fails, the agent reads the output and tries again, and you find out about none of it. Green CI first time, or at the very least the common problems caught before a human is involved.

If you take one thing from this article, take this section.

I use: [https://lefthook.dev/](https://lefthook.dev/)

Worth having for the usual reason, that all code should look like one person wrote it. There is a second reason with agents: formatting noise is the enemy of skim reading, and skim reading is what you are now doing. Every gratuitous whitespace change is a line of diff competing for attention with a line that actually changed behaviour.

Cheapest item on the list too, since most formatters auto correct without the agent needing to do anything.

I use: https://github.com/rubocop/rubocop, https://prettier.io Context is the scarce resource. Filling it with only what is needed is often the difference between an agent that acts smart and one that acts stupid, and the usual failure is not too little documentation but too much, loaded up front, crowding out the thing that mattered.

So having a way to inject context only at the point the agent needs it is worth building. I could not find one that worked the way I wanted, so I wrote agent-apropos, which does just in time context injection across Claude Code, OpenCode, Gemini CLI and Copilot CLI.

I use: [https://github.com/NEXL-LTS/agent-apropos](https://github.com/NEXL-LTS/agent-apropos)

If you have a practice that is non standard, or nuanced enough that no off the shelf tool encodes it, write your own. Most linters let you extend them with custom rules, and writing a standalone checker has never been easier, not least because you can get an agent to write it.

This is also the highest leverage item here over time. The off the shelf tools encode what everyone knows. Custom checkers encode what your team knows, which is the part an agent has no other way of learning.

Worth being straight about the trade offs, because they are real.

Mutation testing is slow. It is not something you run on every commit on a large suite, and you will need to think about where in the workflow it fits.

100% coverage can buy you test shaped noise if you are not careful, tests written to satisfy the threshold rather than to describe behaviour. Mutation testing is the counterweight, which is part of why those two belong together.

Hooks that take too long will get bypassed, by you as much as by the agent. Keep the pre-commit set fast and push the slow things later.

None of that changes the conclusion. It just means picking the order. If you are starting from nothing: hooks and a formatter first, because they cost almost nothing and make everything after them enforceable. Then coverage, then security. Mutation testing when you can afford it.

Hopefully you, or your agent, found this useful.

── more in #artificial-intelligence 4 stories · sorted by recency
── more on @rspec 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
Live at https://your-agent.zahid.host
Get free account → Pricing
from €0/mo · no card required
LIVE [news/tooling-every-ai-sof…] indexed:0 read:8min 2026-08-22 ·