Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill

Tadashi Shigeoka added a skill called `omh-pr-multi-review` to the oh-my-hermes repository (PR #5) that automates running both Claude Code and Codex on the same pull request to catch blind spots each model misses. The skill runs a deterministic Python script that validates the PR URL, checks out the branch, invokes four review commands (claude /review, /security-review, /code-review, and codex review), aggregates the results, and writes a Markdown report. Shigeoka argues that keeping the procedure in a script rather than natural-language steps prevents the agent from skipping steps or making dangerous git commands.

Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill Tadashi Shigeoka /en/author/tadashi-shigeoka/ · Wed, July 22, 2026 Leave a pull request review to a single model and whatever that model structurally overlooks stays overlooked forever. Ask the same model twice and it walks past the same spot twice. The obvious countermeasure is to have models from different lineages read the same diff. Claude Code https://code.claude.com/docs/en/overview and Codex https://openai.com/codex/ differ in both training data and post-training, so their blind spots sit in different places. Anything both flag independently is very likely real. Anything only one of them raises might be useful or might be noise, and that is exactly the set a human should adjudicate. I had been doing exactly that already: several terminal tabs, Claude Code and Codex each in their own session, then reading the results side by side. But walking through that by hand on every PR is tedious, and it is easy to forget to run one of them. Collapsing the same routine into a single command is where this started. So I added a skill called omh-pr-multi-review to oh-my-hermes https://github.com/codenote-net/oh-my-hermes PR 5 https://github.com/codenote-net/oh-my-hermes/pull/5 . This post covers its design and the implementation decisions behind it. About the oh-my-hermes repository For context, oh-my-hermes is a collection of personal customizations for Hermes Agent https://github.com/NousResearch/hermes-agent . The rule is that the agent core is never modified: only skills, plugins, hooks, and config templates are bolted on from outside. Every identifier carries an omh prefix so nothing collides with other extensions. Skills are distributable as a tap. The new omh-pr-multi-review is one skill under that skills directory. It is only three files. Don’t let the agent narrate the procedure The first decision was to keep the review procedure out of the agent’s judgment entirely. When people hear “skill,” they usually picture a SKILL.md full of natural-language steps that the agent reads and executes in order. That is certainly easier to write. But for work with strong side effects like this switch the checkout, invoke four CLIs in sequence, always return to the original branch , that shape is dangerous. Agents skip steps, and when something fails they may helpfully reach for something like git checkout -f . So SKILL.md says essentially one thing: run the bundled script. All the actual control flow lives in review pr.py . The only discretion left to the agent is passing the PR URL and optionally overriding the output path or the timeout. Draw the boundary between the deterministic script and the nondeterministic agent as early as possible. I think that generalizes well beyond this case, to any skill that carries side effects. The overall flow php flowchart TD START Receive PR URL -- VAL Validate URL format, CLI presence, auth,<br/ repository identity, clean working tree VAL -- |fail| ABORT Abort with exit 2 VAL -- |pass| CO gh pr checkout --detach CO -- R1 claude /review CO -- R2 claude /security-review CO -- R3 claude /code-review CO -- R4 codex review R1 -- AGG Aggregate four sections R2 -- AGG R3 -- AGG R4 -- AGG AGG -- SUM Generate cross-review summary<br/ via codex exec SUM -- OUT Write Markdown report AGG -- RESTORE Restore original checkout Front-load the validation Before doing anything else, the script checks every precondition it can. A failure caught here is vastly cheaper than the same failure caught after thirty minutes of reviewing. The PR URL is matched strictly against a regular expression. It uses fullmatch , so nothing sneaks through on a partial match. Query strings and fragments which browsers love to attach when you copy a URL are tolerated. Next, every required command must be on PATH . Then authentication. Each of the three CLIs gets a login check, and a failure aborts with a concrete remedy attached. Putting “here is what to do next” into the error message matters more than usual for tools an agent drives. Given an explicit remedy, an agent fixes the problem itself; given a bare exit 1 , it starts guessing and doing strange things. Two more checks are quiet but important. The first confirms that the current repository matches the PR’s repository. Running gh pr checkout with a PR URL from a different repository leaves you in an incoherent state. GitHub owner and repository names are case-insensitive, so the comparison lowercases both sides. The second is working tree cleanliness. With --untracked-files=all , even a stray untracked file stops the run. That may look overly strict, but this script switches branches, and I did not want to leave any path where a user’s local changes get caught up in that. The script never force-checks-out and never discards changes. The reviewer lineup Once the checkout lands, four reviewers run in sequence. The Claude Code side invokes three slash commands with identical model settings. -p runs non-interactively, --effort high buys deeper reasoning, and --no-session-persistence leaves no session history behind. The three commands are kept separate because /review is general review, /security-review takes a security lens, and /code-review does a detailed pass over the diff, each with its own prompt and perspective. Even within one model, changing the lens changes what gets caught. The Codex side injects its settings inline. Configuration goes through -c because I do not want to touch the user’s config.toml . A tool that mutates global configuration changes the state of your environment after a single use, and the accident shows up later, while you are doing something else. service tier="fast" selects the Fast tier in the current CLI, which noticeably shortens the wait. The comparison base passed to --base resolves to origin/<base using the baseRefName read from gh pr view . If the fetch fails, it falls back to the local <base and warns on stderr. A stale local base inflates the diff and blurs the review, so a successful fetch is what you want. But degrading is better than refusing to run offline, so it continues in a degraded mode. Isolate the failures Failure handling took the most care in this script. Consider three reviews succeeding over thirty minutes and the fourth timing out. Failing the whole run and keeping nothing would be the worst possible design. The three successes have value, and the fact that the fourth died is itself information. So per-reviewer results accumulate as tuples rather than raising. Timeouts work the same way: partial output is rescued from subprocess.TimeoutExpired and kept in the report. Findings that made it out before the cutoff are worth reading on their own. The exit code is 124 to match the timeout 1 convention. If the checkout itself fails, all four sections are filled with the same error and the report is still written. A report explaining why it is empty beats an empty report. Restoration always runs in a finally block. The restore target is the branch name recorded before the run. If the script was started from a detached HEAD, symbolic-ref fails and it falls back to the commit SHA. Even when restoration fails, the report is written and a warning is printed. Raising there and throwing the report away would just double the user’s loss. Generating the cross-review summary Once all four results are in, the model gets called one last time. This is the point of the whole skill. Three things matter in that instruction. - Only pick up items that multiple reviewers flagged independently. Solo findings stay out of the summary. - Merge duplicates, and name which reviewers agreed. - If there is no credible overlap, say so. The third is the critical one. Tell a summarizing model to “summarize” and it will produce something plausible-sounding even with nothing to work from. Unless you explicitly authorize the conclusion “there is no overlap,” the summary becomes a machine that always lies a little. The call is read-only and ephemeral. --sandbox read-only removes file writes and --ephemeral leaves no session. There is no reason for the summarization phase to touch code, so the capability is dropped entirely. The review results are written to a Markdown file in a temporary directory and only the path is passed, keeping a large blob of text off the command line. If summary generation fails, the report as a whole is still a success. The summary is a way in, not the evidence. The evidence is the raw output in each section. That ordering is written into SKILL.md’s output contract as well: do not present the generated summary as a replacement for the individual evidence. The shape of the report The final output looks like this. The UTC execution timestamp and the exact model options are recorded because rereading a report later is useless if you cannot tell which configuration produced it. Models get replaced within months. Judging the quality of findings in a six-month-old report requires knowing what was running at the time. Practical impressions and cost The obvious cost is four high-effort inference runs per PR, plus one more for the summary. That is not cheap, especially with both --effort high and model reasoning effort="high" set. So this is not something to run on every PR. It fits situations like these. - PRs touching authentication, authorization, billing, or anywhere a mistake is expensive. - Code that handles external input or spawns subprocesses. - PRs you did not write yourself, or that an agent wrote a lot of, where you do not fully hold the shape in your head. Conversely, throwing four models’ worth of reasoning at a dependency bump, a config value change, or a typo fix is simply waste. A single review remains the right amount for those. There is also a wall-clock issue. The four reviews run serially, so you can wait through the default 1,800-second timeout four times over. I considered parallelizing, but that has four processes reading the same working tree at once, plus tangled output and rate-limit handling, so the first version stays serial. Splitting the working trees with git worktree https://git-scm.com/docs/git-worktree would resolve it, and that remains on the list. Conclusion A code review from a single model inherits that model’s blind spots wholesale. Have models from different lineages read the same diff independently, and look first at where their findings overlap. omh-pr-multi-review makes that simple workflow reproducible in one command. Three things drove the implementation. Confine procedural decisions to a deterministic script rather than the agent. Isolate reviewer failures so successful results always survive. And position the cross-review summary as a way in rather than a substitute for raw output, with explicit permission to report that no overlap exists. The three prohibitions never mutate global configuration, never force checkout, never run against a dirty working tree mattered just as much for a tool an agent invokes. For a human-operated tool, “please be careful” would cover it. When an agent is the caller, the only real guarantee is not implementing the dangerous path at all. That’s all from building a Hermes Agent skill that has Claude Code and Codex review the same PR independently and cross-checks their findings, from the Gemba.