Every week someone asks which coding agent is better, and every answer is a vibe. I wanted a diff.
So I built pairmark. One command in a JavaScript or TypeScript repo:
npx pairmark "add rate limiting to POST /api/login"
It creates two git worktrees at the same commit, gives Claude Code and Codex the identical brief, and streams both live in a split screen. When they finish, your own checks run in each worktree: tests, typecheck, lint, whatever package.json
has. Then each agent reviews both patches blind, labelled A and B in random order, and scores correctness, completeness, quality and safety with citations. You get one HTML file with the verdict, the rule that decided it, both diffs, and a receipt image sized for posting.
Two of those lines came out of an argument. I asked Codex to attack the first design before writing any code. Its second-round critique said a patch could win the checks by weakening them, and that blind judging by the same two models is only partly blind, since a model can recognise its own style. Both points are in the rules: a patch that edits package.json
, tsconfig
, or any test or lint configuration is flagged and cannot win on the checks alone, and judges only decide when they agree. A split is never averaged into a winner, because a 0.3 gap between two models grading their own work blind is noise, and calling noise a winner is the fastest way to make a tool untrustworthy. The whole discussion ships in the repo, verbatim.
I asked both agents to add a runs
subcommand to pairmark itself: list past runs as a table, newest first, with a unit test.
| Claude Code | Codex | |
|---|---|---|
| Time | 3m 28s | 5m 15s |
| Files changed | 4 (+122, -3) | 5 (+105, -3) |
| Shell commands | 11 | 37 |
| Context read | 548K tokens | 1.23M tokens |
| typecheck, test, build | pass, pass, pass | pass, pass, pass |
| Judge score, avg of two blind judges | 8.6 / 10 | 8.0 / 10 |
Both green. Then the twist. Codex also noticed that my build script would fail on a fresh checkout, because it copied a file into a directory that did not exist yet, and it fixed that. The fix lives in package.json
. Rule 2 says a patch that touches the check configuration cannot win on the checks alone, so the win went to Claude Code, and Codex's dissent was recorded on the report: as a judge, Codex would have picked its own patch for matching the requested columns exactly.
That is the tool working as designed, against the agent that arguably did more. I took Codex's fix by hand and shipped Claude Code's patch as the real runs
command. You can run it today.
On a demo repo, the task was input validation with a typed error and tests. Both passed. Both judges, reviewing blind, picked their own patch. The rule says that is a tie, and the report says why each judge leaned the way it did: Claude Code's judge valued the broader tests, Codex's judge valued the smaller diff.
I find that more useful than a number. The same task produced two defensible answers, and the report tells you which values produced which answer.
It is not a benchmark. It says "on this task, in this repo, today", and it says the rule that decided it. Run it ten times on ten real tasks and you have a basis for a decision about your own codebase. Run it once and you have a receipt.
It also cannot remove model preference from judging. That is why checks outrank judges, why judging is cross-checked, and why the report shows each judge's pick and reason instead of a single score.
node_modules
is cloned into each worktree with copy-on-write where the filesystem allows, so both agents get their own tree in milliseconds.Source: https://github.com/Hemanshu-Upadhyay/pairmark
If you run it on your repo, I would like to see the receipt. And if there is a repo you cannot share, send me the task and I will run it and send you the report.