# Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill

> Source: <https://codenote.net/en/posts/multi-model-pr-review-claude-code-codex-hermes-agent-skill/>
> Published: 2026-07-22 14:52:39+00:00

# Reviewing a PR with Claude Code and Codex at Once — Automating Multi-Model Review as a Hermes Agent Skill

[Tadashi Shigeoka](/en/author/tadashi-shigeoka/)· Wed, July 22, 2026

Leave a pull request review to a single model and whatever that model structurally overlooks stays overlooked forever. Ask the same model twice and it walks past the same spot twice.

The obvious countermeasure is to have models from different lineages read the same diff. [Claude Code](https://code.claude.com/docs/en/overview) and [Codex](https://openai.com/codex/) differ in both training data and post-training, so their blind spots sit in different places. Anything both flag independently is very likely real. Anything only one of them raises might be useful or might be noise, and that is exactly the set a human should adjudicate.

I had been doing exactly that already: several terminal tabs, Claude Code and Codex each in their own session, then reading the results side by side. But walking through that by hand on every PR is tedious, and it is easy to forget to run one of them. Collapsing the same routine into a single command is where this started.

So I added a skill called `omh-pr-multi-review`

to [oh-my-hermes](https://github.com/codenote-net/oh-my-hermes) ([PR #5](https://github.com/codenote-net/oh-my-hermes/pull/5)). This post covers its design and the implementation decisions behind it.

## About the oh-my-hermes repository

For context, oh-my-hermes is a collection of personal customizations for [Hermes Agent](https://github.com/NousResearch/hermes-agent). The rule is that the agent core is never modified: only skills, plugins, hooks, and config templates are bolted on from outside. Every identifier carries an `omh`

prefix so nothing collides with other extensions.

Skills are distributable as a tap.

The new `omh-pr-multi-review`

is one skill under that skills directory. It is only three files.

## Don’t let the agent narrate the procedure

The first decision was to keep the review procedure out of the agent’s judgment entirely.

When people hear “skill,” they usually picture a SKILL.md full of natural-language steps that the agent reads and executes in order. That is certainly easier to write. But for work with strong side effects like this (switch the checkout, invoke four CLIs in sequence, always return to the original branch), that shape is dangerous. Agents skip steps, and when something fails they may helpfully reach for something like `git checkout -f`

.

So SKILL.md says essentially one thing: run the bundled script.

All the actual control flow lives in `review_pr.py`

. The only discretion left to the agent is passing the PR URL and optionally overriding the output path or the timeout.

Draw the boundary between the deterministic script and the nondeterministic agent as early as possible. I think that generalizes well beyond this case, to any skill that carries side effects.

## The overall flow

``` php
flowchart TD
    START[Receive PR URL] --> VAL[Validate URL format, CLI presence, auth,<br/>repository identity, clean working tree]
    VAL -->|fail| ABORT[Abort with exit 2]
    VAL -->|pass| CO[gh pr checkout --detach]
    CO --> R1[claude /review]
    CO --> R2[claude /security-review]
    CO --> R3[claude /code-review]
    CO --> R4[codex review]
    R1 --> AGG[Aggregate four sections]
    R2 --> AGG
    R3 --> AGG
    R4 --> AGG
    AGG --> SUM[Generate cross-review summary<br/>via codex exec]
    SUM --> OUT[Write Markdown report]
    AGG --> RESTORE[Restore original checkout]
```

## Front-load the validation

Before doing anything else, the script checks every precondition it can. A failure caught here is vastly cheaper than the same failure caught after thirty minutes of reviewing.

The PR URL is matched strictly against a regular expression.

It uses `fullmatch`

, so nothing sneaks through on a partial match. Query strings and fragments (which browsers love to attach when you copy a URL) are tolerated.

Next, every required command must be on `PATH`

.

Then authentication. Each of the three CLIs gets a login check, and a failure aborts with a concrete remedy attached.

Putting “here is what to do next” into the error message matters more than usual for tools an agent drives. Given an explicit remedy, an agent fixes the problem itself; given a bare `exit 1`

, it starts guessing and doing strange things.

Two more checks are quiet but important.

The first confirms that the current repository matches the PR’s repository.

Running `gh pr checkout`

with a PR URL from a different repository leaves you in an incoherent state. GitHub owner and repository names are case-insensitive, so the comparison lowercases both sides.

The second is working tree cleanliness.

With `--untracked-files=all`

, even a stray untracked file stops the run. That may look overly strict, but this script switches branches, and I did not want to leave any path where a user’s local changes get caught up in that. The script never force-checks-out and never discards changes.

## The reviewer lineup

Once the checkout lands, four reviewers run in sequence. The Claude Code side invokes three slash commands with identical model settings.

`-p`

runs non-interactively, `--effort high`

buys deeper reasoning, and `--no-session-persistence`

leaves no session history behind. The three commands are kept separate because `/review`

is general review, `/security-review`

takes a security lens, and `/code-review`

does a detailed pass over the diff, each with its own prompt and perspective. Even within one model, changing the lens changes what gets caught.

The Codex side injects its settings inline.

Configuration goes through `-c`

because I do not want to touch the user’s `config.toml`

. A tool that mutates global configuration changes the state of your environment after a single use, and the accident shows up later, while you are doing something else. `service_tier="fast"`

selects the Fast tier in the current CLI, which noticeably shortens the wait.

The comparison base passed to `--base`

resolves to `origin/<base>`

using the `baseRefName`

read from `gh pr view`

. If the fetch fails, it falls back to the local `<base>`

and warns on stderr.

A stale local base inflates the diff and blurs the review, so a successful fetch is what you want. But degrading is better than refusing to run offline, so it continues in a degraded mode.

## Isolate the failures

Failure handling took the most care in this script.

Consider three reviews succeeding over thirty minutes and the fourth timing out. Failing the whole run and keeping nothing would be the worst possible design. The three successes have value, and the fact that the fourth died is itself information.

So per-reviewer results accumulate as tuples rather than raising.

Timeouts work the same way: partial output is rescued from `subprocess.TimeoutExpired`

and kept in the report.

Findings that made it out before the cutoff are worth reading on their own. The exit code is 124 to match the `timeout(1)`

convention.

If the checkout itself fails, all four sections are filled with the same error and the report is still written. A report explaining why it is empty beats an empty report.

Restoration always runs in a `finally`

block.

The restore target is the branch name recorded before the run. If the script was started from a detached HEAD, `symbolic-ref`

fails and it falls back to the commit SHA.

Even when restoration fails, the report is written and a warning is printed. Raising there and throwing the report away would just double the user’s loss.

## Generating the cross-review summary

Once all four results are in, the model gets called one last time. This is the point of the whole skill.

Three things matter in that instruction.

- Only pick up items that multiple reviewers flagged independently. Solo findings stay out of the summary.
- Merge duplicates, and name which reviewers agreed.
- If there is no credible overlap, say so.

The third is the critical one. Tell a summarizing model to “summarize” and it will produce something plausible-sounding even with nothing to work from. Unless you explicitly authorize the conclusion “there is no overlap,” the summary becomes a machine that always lies a little.

The call is read-only and ephemeral.

`--sandbox read-only`

removes file writes and `--ephemeral`

leaves no session. There is no reason for the summarization phase to touch code, so the capability is dropped entirely. The review results are written to a Markdown file in a temporary directory and only the path is passed, keeping a large blob of text off the command line.

If summary generation fails, the report as a whole is still a success.

The summary is a way in, not the evidence. The evidence is the raw output in each section. That ordering is written into SKILL.md’s output contract as well: do not present the generated summary as a replacement for the individual evidence.

## The shape of the report

The final output looks like this.

The UTC execution timestamp and the exact model options are recorded because rereading a report later is useless if you cannot tell which configuration produced it. Models get replaced within months. Judging the quality of findings in a six-month-old report requires knowing what was running at the time.

## Practical impressions and cost

The obvious cost is four high-effort inference runs per PR, plus one more for the summary. That is not cheap, especially with both `--effort high`

and `model_reasoning_effort="high"`

set.

So this is not something to run on every PR. It fits situations like these.

- PRs touching authentication, authorization, billing, or anywhere a mistake is expensive.
- Code that handles external input or spawns subprocesses.
- PRs you did not write yourself, or that an agent wrote a lot of, where you do not fully hold the shape in your head.

Conversely, throwing four models’ worth of reasoning at a dependency bump, a config value change, or a typo fix is simply waste. A single review remains the right amount for those.

There is also a wall-clock issue. The four reviews run serially, so you can wait through the default 1,800-second timeout four times over. I considered parallelizing, but that has four processes reading the same working tree at once, plus tangled output and rate-limit handling, so the first version stays serial. Splitting the working trees with [git worktree](https://git-scm.com/docs/git-worktree) would resolve it, and that remains on the list.

## Conclusion

A code review from a single model inherits that model’s blind spots wholesale. Have models from different lineages read the same diff independently, and look first at where their findings overlap. `omh-pr-multi-review`

makes that simple workflow reproducible in one command.

Three things drove the implementation. Confine procedural decisions to a deterministic script rather than the agent. Isolate reviewer failures so successful results always survive. And position the cross-review summary as a way in rather than a substitute for raw output, with explicit permission to report that no overlap exists.

The three prohibitions (never mutate global configuration, never force checkout, never run against a dirty working tree) mattered just as much for a tool an agent invokes. For a human-operated tool, “please be careful” would cover it. When an agent is the caller, the only real guarantee is not implementing the dangerous path at all.

That’s all from building a Hermes Agent skill that has Claude Code and Codex review the same PR independently and cross-checks their findings, from the Gemba.