Your Tests Pass. So Does the Wrong Code

A developer built TestSlop, a tool that takes a code diff and finds a nearby alternative implementation—an "Evil Twin"—that behaves differently but still passes the same test suite, then reports the input that distinguishes the two. In one replay of a real `ms` library commit that added month formatting, the original and a variant with `msAbs >= y` changed to `msAbs > y` both passed 163 of 163 tests, with the missing witness being exactly one year (31,557,600,000 ms), where output shifts from `1y` to `12mo`. The tool is limited to JavaScript and TypeScript boundary comparisons on changed lines and is available on GitHub as a local build, not published to npm.

A green test suite tells you one thing: the implementation you wrote satisfies the tests you wrote. It doesn't tell you whether the version one small edit away would pass too. If swapping < for <= would have sailed through, the suite was never checking that boundary. Coding agents make this sharper. They write the code and the tests from the same assumption, the suite goes green, and nobody stops to ask what else it would accept. So I built TestSlop. Point it at a diff and it finds one nearby alternative implementation your tests also accept, then shows the input that tells the two apart. One sentence: TestSlop finds one nearby "Evil Twin" implementation that behaves differently but still passes the same tests. That GIF runs a three-line quantity check. As text: ORIGINAL quantity <= 0 EVIL TWIN quantity <= 1 ORIGINAL 3 / 3 tests passed EVIL TWIN 3 / 3 tests passed Missing witness: quantity = 1 Both passed. The two versions only disagree at quantity = 1 , and no test covers that value. An Evil Twin is a nearby implementation that behaves differently but still passes the same tests. The missing witness is the input that separates them. Here it is quantity = 1 , the simplest input where the two versions give different answers. TestSlop hands you the pair and the witness. Which version matches your contract stays your call. Most of this repo's public face went through an agent workflow. This slice is the part that pushed the sanitized history to GitHub and checked the rendered README, then returned a release verdict: The diagram is the whole loop: shift one boundary comparison, rerun your tests against the alternative in a scratch copy, and print both results with the witness. Your source tree stays untouched. One fixture in the repo replays a real ms commit, the one that added month formatting, against its preserved test suite. The original and the Evil Twin msAbs = y changed to msAbs y both passed 163 of 163 tests. The missing witness is exactly one year: at 31,557,600,000 ms the output moves from 1y to 12mo . The formatter tests checked one millisecond past a year and never exactly one year. One historical example, replayable offline. It doesn't show that TestSlop catches shipped bugs in general. TestSlop is narrow on purpose. It looks at JavaScript and TypeScript boundary comparisons on changed lines, so plenty of diffs produce no Twin at all. The code is on GitHub: github.com/hyukvoid/TestSlop https://github.com/hyukvoid/TestSlop It runs from a local build npm install , then npm run demo . It isn't published to npm. The repo keeps the research notes from the three phases that narrowed the project down to this idea. If you've used mutation testing or coding-agent workflows, I'd be curious whether one concrete counterexample feels more useful than a score.