{"slug": "i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did", "title": "I made Claude Code and Codex race on my repo. The rule went against the agent that did more.", "summary": "A developer built pairmark, a tool that races Claude Code and Codex on the same coding task in isolated git worktrees, then has both agents blind-judge each other's patches. In a self-test, Claude Code won despite Codex fixing a build bug, because pairmark's rules penalize patches that alter check configurations. The tool aims to provide reproducible comparisons of coding agents on real repos.", "body_md": "Every week someone asks which coding agent is better, and every answer is a vibe. I wanted a diff.\n\nSo I built pairmark. One command in a JavaScript or TypeScript repo:\n\n```\nnpx pairmark \"add rate limiting to POST /api/login\"\n```\n\nIt creates two git worktrees at the same commit, gives Claude Code and Codex the identical brief, and streams both live in a split screen. When they finish, your own checks run in each worktree: tests, typecheck, lint, whatever `package.json`\n\nhas. Then each agent reviews both patches blind, labelled A and B in random order, and scores correctness, completeness, quality and safety with citations. You get one HTML file with the verdict, the rule that decided it, both diffs, and a receipt image sized for posting.\n\nTwo of those lines came out of an argument. I asked Codex to attack the first design before writing any code. Its second-round critique said a patch could win the checks by weakening them, and that blind judging by the same two models is only partly blind, since a model can recognise its own style. Both points are in the rules: a patch that edits `package.json`\n\n, `tsconfig`\n\n, or any test or lint configuration is flagged and cannot win on the checks alone, and judges only decide when they agree. A split is never averaged into a winner, because a 0.3 gap between two models grading their own work blind is noise, and calling noise a winner is the fastest way to make a tool untrustworthy. The whole discussion ships in the repo, verbatim.\n\nI asked both agents to add a `runs`\n\nsubcommand to pairmark itself: list past runs as a table, newest first, with a unit test.\n\n| Claude Code | Codex | |\n|---|---|---|\n| Time | 3m 28s | 5m 15s |\n| Files changed | 4 (+122, -3) | 5 (+105, -3) |\n| Shell commands | 11 | 37 |\n| Context read | 548K tokens | 1.23M tokens |\n| typecheck, test, build | pass, pass, pass | pass, pass, pass |\n| Judge score, avg of two blind judges | 8.6 / 10 | 8.0 / 10 |\n\nBoth green. Then the twist. Codex also noticed that my build script would fail on a fresh checkout, because it copied a file into a directory that did not exist yet, and it fixed that. The fix lives in `package.json`\n\n. Rule 2 says a patch that touches the check configuration cannot win on the checks alone, so the win went to Claude Code, and Codex's dissent was recorded on the report: as a judge, Codex would have picked its own patch for matching the requested columns exactly.\n\nThat is the tool working as designed, against the agent that arguably did more. I took Codex's fix by hand and shipped Claude Code's patch as the real `runs`\n\ncommand. You can run it today.\n\nOn a demo repo, the task was input validation with a typed error and tests. Both passed. Both judges, reviewing blind, picked their own patch. The rule says that is a tie, and the report says why each judge leaned the way it did: Claude Code's judge valued the broader tests, Codex's judge valued the smaller diff.\n\nI find that more useful than a number. The same task produced two defensible answers, and the report tells you which values produced which answer.\n\nIt is not a benchmark. It says \"on this task, in this repo, today\", and it says the rule that decided it. Run it ten times on ten real tasks and you have a basis for a decision about your own codebase. Run it once and you have a receipt.\n\nIt also cannot remove model preference from judging. That is why checks outrank judges, why judging is cross-checked, and why the report shows each judge's pick and reason instead of a single score.\n\n`node_modules`\n\nis cloned into each worktree with copy-on-write where the filesystem allows, so both agents get their own tree in milliseconds.Source: [https://github.com/Hemanshu-Upadhyay/pairmark](https://github.com/Hemanshu-Upadhyay/pairmark)\n\nIf you run it on your repo, I would like to see the receipt. And if there is a repo you cannot share, send me the task and I will run it and send you the report.", "url": "https://wpnews.pro/news/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did", "canonical_source": "https://dev.to/upadhyayhari_/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-that-did-more-2b8j", "published_at": "2026-09-02 08:57:20+00:00", "updated_at": "2026-09-02 09:23:13.569402+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "artificial-intelligence"], "entities": ["Claude Code", "Codex", "pairmark", "Hemanshu-Upadhyay"], "alternates": {"html": "https://wpnews.pro/news/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did", "markdown": "https://wpnews.pro/news/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did.md", "text": "https://wpnews.pro/news/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did.txt", "jsonld": "https://wpnews.pro/news/i-made-claude-code-and-codex-race-on-my-repo-the-rule-went-against-the-agent-did.jsonld"}}