{"slug": "reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review", "title": "Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill", "summary": "Tadashi Shigeoka added a skill called `omh-pr-multi-review` to the oh-my-hermes repository (PR #5) that automates running both Claude Code and Codex on the same pull request to catch blind spots each model misses. The skill runs a deterministic Python script that validates the PR URL, checks out the branch, invokes four review commands (claude /review, /security-review, /code-review, and codex review), aggregates the results, and writes a Markdown report. Shigeoka argues that keeping the procedure in a script rather than natural-language steps prevents the agent from skipping steps or making dangerous git commands.", "body_md": "# Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill\n\n[Tadashi Shigeoka](/en/author/tadashi-shigeoka/)· Wed, July 22, 2026\n\nLeave a pull request review to a single model and whatever that model structurally overlooks stays overlooked forever. Ask the same model twice and it walks past the same spot twice.\n\nThe obvious countermeasure is to have models from different lineages read the same diff. [Claude Code](https://code.claude.com/docs/en/overview) and [Codex](https://openai.com/codex/) differ in both training data and post-training, so their blind spots sit in different places. Anything both flag independently is very likely real. Anything only one of them raises might be useful or might be noise, and that is exactly the set a human should adjudicate.\n\nI had been doing exactly that already: several terminal tabs, Claude Code and Codex each in their own session, then reading the results side by side. But walking through that by hand on every PR is tedious, and it is easy to forget to run one of them. Collapsing the same routine into a single command is where this started.\n\nSo I added a skill called `omh-pr-multi-review`\n\nto [oh-my-hermes](https://github.com/codenote-net/oh-my-hermes) ([PR #5](https://github.com/codenote-net/oh-my-hermes/pull/5)). This post covers its design and the implementation decisions behind it.\n\n## About the oh-my-hermes repository\n\nFor context, oh-my-hermes is a collection of personal customizations for [Hermes Agent](https://github.com/NousResearch/hermes-agent). The rule is that the agent core is never modified: only skills, plugins, hooks, and config templates are bolted on from outside. Every identifier carries an `omh`\n\nprefix so nothing collides with other extensions.\n\nSkills are distributable as a tap.\n\nThe new `omh-pr-multi-review`\n\nis one skill under that skills directory. It is only three files.\n\n## Don’t let the agent narrate the procedure\n\nThe first decision was to keep the review procedure out of the agent’s judgment entirely.\n\nWhen people hear “skill,” they usually picture a SKILL.md full of natural-language steps that the agent reads and executes in order. That is certainly easier to write. But for work with strong side effects like this (switch the checkout, invoke four CLIs in sequence, always return to the original branch), that shape is dangerous. Agents skip steps, and when something fails they may helpfully reach for something like `git checkout -f`\n\n.\n\nSo SKILL.md says essentially one thing: run the bundled script.\n\nAll the actual control flow lives in `review_pr.py`\n\n. The only discretion left to the agent is passing the PR URL and optionally overriding the output path or the timeout.\n\nDraw the boundary between the deterministic script and the nondeterministic agent as early as possible. I think that generalizes well beyond this case, to any skill that carries side effects.\n\n## The overall flow\n\n``` php\nflowchart TD\n    START[Receive PR URL] --> VAL[Validate URL format, CLI presence, auth,<br/>repository identity, clean working tree]\n    VAL -->|fail| ABORT[Abort with exit 2]\n    VAL -->|pass| CO[gh pr checkout --detach]\n    CO --> R1[claude /review]\n    CO --> R2[claude /security-review]\n    CO --> R3[claude /code-review]\n    CO --> R4[codex review]\n    R1 --> AGG[Aggregate four sections]\n    R2 --> AGG\n    R3 --> AGG\n    R4 --> AGG\n    AGG --> SUM[Generate cross-review summary<br/>via codex exec]\n    SUM --> OUT[Write Markdown report]\n    AGG --> RESTORE[Restore original checkout]\n```\n\n## Front-load the validation\n\nBefore doing anything else, the script checks every precondition it can. A failure caught here is vastly cheaper than the same failure caught after thirty minutes of reviewing.\n\nThe PR URL is matched strictly against a regular expression.\n\nIt uses `fullmatch`\n\n, so nothing sneaks through on a partial match. Query strings and fragments (which browsers love to attach when you copy a URL) are tolerated.\n\nNext, every required command must be on `PATH`\n\n.\n\nThen authentication. Each of the three CLIs gets a login check, and a failure aborts with a concrete remedy attached.\n\nPutting “here is what to do next” into the error message matters more than usual for tools an agent drives. Given an explicit remedy, an agent fixes the problem itself; given a bare `exit 1`\n\n, it starts guessing and doing strange things.\n\nTwo more checks are quiet but important.\n\nThe first confirms that the current repository matches the PR’s repository.\n\nRunning `gh pr checkout`\n\nwith a PR URL from a different repository leaves you in an incoherent state. GitHub owner and repository names are case-insensitive, so the comparison lowercases both sides.\n\nThe second is working tree cleanliness.\n\nWith `--untracked-files=all`\n\n, even a stray untracked file stops the run. That may look overly strict, but this script switches branches, and I did not want to leave any path where a user’s local changes get caught up in that. The script never force-checks-out and never discards changes.\n\n## The reviewer lineup\n\nOnce the checkout lands, four reviewers run in sequence. The Claude Code side invokes three slash commands with identical model settings.\n\n`-p`\n\nruns non-interactively, `--effort high`\n\nbuys deeper reasoning, and `--no-session-persistence`\n\nleaves no session history behind. The three commands are kept separate because `/review`\n\nis general review, `/security-review`\n\ntakes a security lens, and `/code-review`\n\ndoes a detailed pass over the diff, each with its own prompt and perspective. Even within one model, changing the lens changes what gets caught.\n\nThe Codex side injects its settings inline.\n\nConfiguration goes through `-c`\n\nbecause I do not want to touch the user’s `config.toml`\n\n. A tool that mutates global configuration changes the state of your environment after a single use, and the accident shows up later, while you are doing something else. `service_tier=\"fast\"`\n\nselects the Fast tier in the current CLI, which noticeably shortens the wait.\n\nThe comparison base passed to `--base`\n\nresolves to `origin/<base>`\n\nusing the `baseRefName`\n\nread from `gh pr view`\n\n. If the fetch fails, it falls back to the local `<base>`\n\nand warns on stderr.\n\nA stale local base inflates the diff and blurs the review, so a successful fetch is what you want. But degrading is better than refusing to run offline, so it continues in a degraded mode.\n\n## Isolate the failures\n\nFailure handling took the most care in this script.\n\nConsider three reviews succeeding over thirty minutes and the fourth timing out. Failing the whole run and keeping nothing would be the worst possible design. The three successes have value, and the fact that the fourth died is itself information.\n\nSo per-reviewer results accumulate as tuples rather than raising.\n\nTimeouts work the same way: partial output is rescued from `subprocess.TimeoutExpired`\n\nand kept in the report.\n\nFindings that made it out before the cutoff are worth reading on their own. The exit code is 124 to match the `timeout(1)`\n\nconvention.\n\nIf the checkout itself fails, all four sections are filled with the same error and the report is still written. A report explaining why it is empty beats an empty report.\n\nRestoration always runs in a `finally`\n\nblock.\n\nThe restore target is the branch name recorded before the run. If the script was started from a detached HEAD, `symbolic-ref`\n\nfails and it falls back to the commit SHA.\n\nEven when restoration fails, the report is written and a warning is printed. Raising there and throwing the report away would just double the user’s loss.\n\n## Generating the cross-review summary\n\nOnce all four results are in, the model gets called one last time. This is the point of the whole skill.\n\nThree things matter in that instruction.\n\n- Only pick up items that multiple reviewers flagged independently. Solo findings stay out of the summary.\n- Merge duplicates, and name which reviewers agreed.\n- If there is no credible overlap, say so.\n\nThe third is the critical one. Tell a summarizing model to “summarize” and it will produce something plausible-sounding even with nothing to work from. Unless you explicitly authorize the conclusion “there is no overlap,” the summary becomes a machine that always lies a little.\n\nThe call is read-only and ephemeral.\n\n`--sandbox read-only`\n\nremoves file writes and `--ephemeral`\n\nleaves no session. There is no reason for the summarization phase to touch code, so the capability is dropped entirely. The review results are written to a Markdown file in a temporary directory and only the path is passed, keeping a large blob of text off the command line.\n\nIf summary generation fails, the report as a whole is still a success.\n\nThe summary is a way in, not the evidence. The evidence is the raw output in each section. That ordering is written into SKILL.md’s output contract as well: do not present the generated summary as a replacement for the individual evidence.\n\n## The shape of the report\n\nThe final output looks like this.\n\nThe UTC execution timestamp and the exact model options are recorded because rereading a report later is useless if you cannot tell which configuration produced it. Models get replaced within months. Judging the quality of findings in a six-month-old report requires knowing what was running at the time.\n\n## Practical impressions and cost\n\nThe obvious cost is four high-effort inference runs per PR, plus one more for the summary. That is not cheap, especially with both `--effort high`\n\nand `model_reasoning_effort=\"high\"`\n\nset.\n\nSo this is not something to run on every PR. It fits situations like these.\n\n- PRs touching authentication, authorization, billing, or anywhere a mistake is expensive.\n- Code that handles external input or spawns subprocesses.\n- PRs you did not write yourself, or that an agent wrote a lot of, where you do not fully hold the shape in your head.\n\nConversely, throwing four models’ worth of reasoning at a dependency bump, a config value change, or a typo fix is simply waste. A single review remains the right amount for those.\n\nThere is also a wall-clock issue. The four reviews run serially, so you can wait through the default 1,800-second timeout four times over. I considered parallelizing, but that has four processes reading the same working tree at once, plus tangled output and rate-limit handling, so the first version stays serial. Splitting the working trees with [git worktree](https://git-scm.com/docs/git-worktree) would resolve it, and that remains on the list.\n\n## Conclusion\n\nA code review from a single model inherits that model’s blind spots wholesale. Have models from different lineages read the same diff independently, and look first at where their findings overlap. `omh-pr-multi-review`\n\nmakes that simple workflow reproducible in one command.\n\nThree things drove the implementation. Confine procedural decisions to a deterministic script rather than the agent. Isolate reviewer failures so successful results always survive. And position the cross-review summary as a way in rather than a substitute for raw output, with explicit permission to report that no overlap exists.\n\nThe three prohibitions (never mutate global configuration, never force checkout, never run against a dirty working tree) mattered just as much for a tool an agent invokes. For a human-operated tool, “please be careful” would cover it. When an agent is the caller, the only real guarantee is not implementing the dangerous path at all.\n\nThat’s all from building a Hermes Agent skill that has Claude Code and Codex review the same PR independently and cross-checks their findings, from the Gemba.", "url": "https://wpnews.pro/news/reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review", "canonical_source": "https://codenote.net/en/posts/multi-model-pr-review-claude-code-codex-hermes-agent-skill/", "published_at": "2026-07-22 14:52:39+00:00", "updated_at": "2026-08-01 18:09:28.984028+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-tools"], "entities": ["Tadashi Shigeoka", "oh-my-hermes", "Hermes Agent", "Claude Code", "Codex", "review_pr.py"], "alternates": {"html": "https://wpnews.pro/news/reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review", "markdown": "https://wpnews.pro/news/reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review.md", "text": "https://wpnews.pro/news/reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review.txt", "jsonld": "https://wpnews.pro/news/reviewing-a-pr-with-claude-code-and-codex-at-once-automating-multi-model-review.jsonld"}}