I made Claude Code and Codex race on my repo. The rule went against the agent that did more.

A developer built pairmark, a tool that races Claude Code and Codex on the same coding task in isolated git worktrees, then has both agents blind-judge each other's patches. In a self-test, Claude Code won despite Codex fixing a build bug, because pairmark's rules penalize patches that alter check configurations. The tool aims to provide reproducible comparisons of coding agents on real repos.

Every week someone asks which coding agent is better, and every answer is a vibe. I wanted a diff. So I built pairmark. One command in a JavaScript or TypeScript repo: npx pairmark "add rate limiting to POST /api/login" It creates two git worktrees at the same commit, gives Claude Code and Codex the identical brief, and streams both live in a split screen. When they finish, your own checks run in each worktree: tests, typecheck, lint, whatever package.json has. Then each agent reviews both patches blind, labelled A and B in random order, and scores correctness, completeness, quality and safety with citations. You get one HTML file with the verdict, the rule that decided it, both diffs, and a receipt image sized for posting. Two of those lines came out of an argument. I asked Codex to attack the first design before writing any code. Its second-round critique said a patch could win the checks by weakening them, and that blind judging by the same two models is only partly blind, since a model can recognise its own style. Both points are in the rules: a patch that edits package.json , tsconfig , or any test or lint configuration is flagged and cannot win on the checks alone, and judges only decide when they agree. A split is never averaged into a winner, because a 0.3 gap between two models grading their own work blind is noise, and calling noise a winner is the fastest way to make a tool untrustworthy. The whole discussion ships in the repo, verbatim. I asked both agents to add a runs subcommand to pairmark itself: list past runs as a table, newest first, with a unit test. | Claude Code | Codex | | |---|---|---| | Time | 3m 28s | 5m 15s | | Files changed | 4 +122, -3 | 5 +105, -3 | | Shell commands | 11 | 37 | | Context read | 548K tokens | 1.23M tokens | | typecheck, test, build | pass, pass, pass | pass, pass, pass | | Judge score, avg of two blind judges | 8.6 / 10 | 8.0 / 10 | Both green. Then the twist. Codex also noticed that my build script would fail on a fresh checkout, because it copied a file into a directory that did not exist yet, and it fixed that. The fix lives in package.json . Rule 2 says a patch that touches the check configuration cannot win on the checks alone, so the win went to Claude Code, and Codex's dissent was recorded on the report: as a judge, Codex would have picked its own patch for matching the requested columns exactly. That is the tool working as designed, against the agent that arguably did more. I took Codex's fix by hand and shipped Claude Code's patch as the real runs command. You can run it today. On a demo repo, the task was input validation with a typed error and tests. Both passed. Both judges, reviewing blind, picked their own patch. The rule says that is a tie, and the report says why each judge leaned the way it did: Claude Code's judge valued the broader tests, Codex's judge valued the smaller diff. I find that more useful than a number. The same task produced two defensible answers, and the report tells you which values produced which answer. It is not a benchmark. It says "on this task, in this repo, today", and it says the rule that decided it. Run it ten times on ten real tasks and you have a basis for a decision about your own codebase. Run it once and you have a receipt. It also cannot remove model preference from judging. That is why checks outrank judges, why judging is cross-checked, and why the report shows each judge's pick and reason instead of a single score. node modules is cloned into each worktree with copy-on-write where the filesystem allows, so both agents get their own tree in milliseconds.Source: https://github.com/Hemanshu-Upadhyay/pairmark https://github.com/Hemanshu-Upadhyay/pairmark If you run it on your repo, I would like to see the receipt. And if there is a repo you cannot share, send me the task and I will run it and send you the report.