{"slug": "i-benchmarked-tdd-guard-it-didn-t-write-better-code", "title": "I benchmarked tdd-guard. It didn't write better code", "summary": "A 54-run benchmark of the tdd-guard Claude Code plugin, which blocks code edits until a failing test exists, found the plugin cost 2 to 4 times more and produced no better code than bare Claude Code, according to the write-up by developer Max Taylor. Across 3 projects (a rate limiter, a CSV parser and an async retry helper), 3 Claude Code configurations and 2 rounds with 3 runs each, judged over 243 rounds, the unconstrained configuration won on test quality, design quality and spec adherence, while tdd-guard and a CLAUDE.md-only setup won on restraint; Taylor reported that test-first discipline can stop an agent from building functionality the spec asks for when no test demands it. Taylor said the plugin's real payoff only shows up once an agent extends an existing codebase, and that on quality alone it is hard to distinguish the enforced plugin from pasting the same rules into CLAUDE.md.", "body_md": "# I benchmarked tdd-guard. It didn't write better code\n\ntdd-guard blocks code edits until a failing test exists. 3 projects, 3 Claude Code configurations, a follow-up round, 54 runs. It didn't write better code, but it did buy consistency.\n\nI benchmarked tdd-guard, a Claude Code plugin that blocks code edits until a failing test exists for them. The short version: it costs 2 to 4x more, it can make the agent skip requirements that no test asks for, and its real payoff only shows up once you start extending an existing codebase.\n\nThis is the long version of [the video breakdown](https://youtu.be/EjtkHa_FH0s).\n\n## 01Testing setup\n\nI used [three small projects](https://github.com/max-taylor/tdd-guard-bench/tree/main/specs), each a different shape:\n\n- rate limiter (state-heavy)\n- CSV parser (transformational)\n- retry helper for async operations (control-flow heavy)\n\nEach one ran under three Claude Code configurations:\n\n- **A, no instruction:** bare Claude Code.\n- **B, CLAUDE.md only:** the tdd-guard rules pasted into`CLAUDE.md` as plain instructions, with nothing enforcing them.\n- **C, tdd-guard:** the full plugin with the same rules as B, wired in as a hook that can deny a`Write` or`Edit` .\n\nB and C use the identical rule set. The only difference is whether the rules are enforced or just requested.\n\nEach project also got a [follow-up prompt](https://github.com/max-taylor/tdd-guard-bench/tree/main/followups) that extended the original spec. A single-shot run can't tell you much about discipline, so I wanted to see whether the plugin held up through a second round of changes.\n\nThat's 3 specs, 3 configurations, 2 rounds, and 3 runs of each: 54 runs total.\n\n## 02Judging\n\nTo grade quality, a script took the source and tests from two runs of the same project and put both into a prompt alongside a rubric, the original spec, and the follow-up spec. The judge picked a winner on four dimensions and explained each call.\n\nLLM judges tend to favour whichever candidate they see first, so I randomised the order on every call.\n\nEach configuration had 3 runs, so comparing two configurations means pitting all 3 runs of one against all 3 of the other: 9 pairs. Doing that for A vs B, A vs C, and B vs C gives 27 pairs per project. Across three projects, with three judge calls per pair to get a majority, that's 243 judging rounds.\n\n## 03Quality results\n\n**Test quality.** A won clearly. The judge liked its longer suites and better edge-case handling. C edged out B, but the judge's reasoning usually came down to C writing one extra test, so it was close.\n\n**Design quality.** A won again, for the same reasons. B and C tied.\n\n**Spec adherence.** This measured how closely each implementation matched the spec. A won because B and C both missed an edge case, and both skipped a piece of functionality the spec explicitly asked for. The TDD rules kept them anchored to the tests they'd already written, and once those passed, they stopped. I checked the test files to confirm: neither B nor C ever wrote a test for the missing case.\n\n**Restraint.** Here the judge penalised the exact thing it had rewarded in the first two categories and called it scope creep. This is where tdd-guard's approach pays off: leaner implementations that do what was asked and nothing more.\n\nSo A wins on test and design quality because the judge likes bigger suites and extra handling, and B and C win on restraint. The spec adherence result is the one worth remembering: test-first discipline can stop an agent from building something the spec asks for, if no test happens to demand it. B and C also tracked each other closely on every dimension, so on quality alone it's hard to say what the plugin gives you over pasting the rules into CLAUDE.md.\n\n## 04Test count\n\nA wrote more tests across the board, which lines up with the test quality result. The rate limiter was the exception, where tdd-guard nearly matched it.\n\n## 05Round 2 tests added\n\nThe follow-up round is where the plugin earns its place. When extending the codebase, C added new tests in every run, with the least variance of the three. A was the opposite: some runs added no new tests at all.\n\nThat's the plugin doing exactly what it claims, forcing the same development cycle every time.\n\n## 06Cost\n\nIn round 1, tdd-guard cost 2.4x more than plain Claude Code on the CSV parser, 3.8x on the rate limiter, and 2.2x on the retry helper. In round 2 that dropped to 2.4x, 2.3x, and 1.9x. Extending an existing codebase is cheaper than bootstrapping a blank one, so the gap narrows.\n\nThat premium is what you pay for test-first discipline.\n\n## 07Verdict\n\ntdd-guard costs 2 to 4x more on a fresh codebase and about 2x when extending one. Its strict rule of \"failing test first, always\" means anything without a test can slip through, including things the spec asked for. Where it earns its cost is ongoing work: the hook forces the same cycle every time, so tests keep growing with the code instead of relying on the agent to remember.\n\n**Repo:** [tdd-guard-bench](https://github.com/max-taylor/tdd-guard-bench) · **Video:** [youtu.be/EjtkHa_FH0s](https://youtu.be/EjtkHa_FH0s) · **tdd-guard plugin:** [nizos/tdd-guard](https://github.com/nizos/tdd-guard)", "url": "https://wpnews.pro/news/i-benchmarked-tdd-guard-it-didn-t-write-better-code", "canonical_source": "https://www.maxtaylor.me/articles/i-benchmarked-tdd-guard-it-didn-t-write-better-code", "published_at": "2026-09-28 00:00:00+00:00", "updated_at": "2026-10-27 14:14:33.159296+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-research"], "entities": ["tdd-guard", "Claude Code", "Max Taylor"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-benchmarked-tdd-guard-it-didn-t-write-better-code", "markdown": "https://wpnews.pro/news/i-benchmarked-tdd-guard-it-didn-t-write-better-code.md", "text": "https://wpnews.pro/news/i-benchmarked-tdd-guard-it-didn-t-write-better-code.txt", "jsonld": "https://wpnews.pro/news/i-benchmarked-tdd-guard-it-didn-t-write-better-code.jsonld"}}