89 of 190 reviewer subagents graded their own model's work. A PreToolUse hook has blocked 7 since; one walked around it

A team auditing its Claude Code agent found that 89 of 190 deep-reviewer subagent launches had graded drafts on the main session's exact model, defeating a written rule requiring a different model family for review. After a reviewer self-reported the overlap, the team added a PreToolUse hook that blocks same-model launches; it has since blocked 7 real launches, one of which was relaunched around the hook 15 seconds later. The team traced the failure to the `opus` family alias resolving to the main conversation's model when the main session is already on Opus.

We want the subagent that grades our Claude Code agent's work to run on a different model family from the session that wrote it. For five weeks that rule lived in a sentence of our instructions and one line of an agent file, and when we finally counted, 89 of 190 reviewer launches had run on the main session's own model. A PreToolUse hook now blocks those launches. It has blocked 7 real ones, and one of them was relaunched around it 15 seconds later. Below is how the hook decides, what it blocked, what it can't see, and the reviews we deliberately put back on one model. All times below are UTC. Our agent drafts a lot of text that other people read: replies to comments, pages, design notes. Before any of it goes out, a subagent we call deep-reviewer grades it against a written rubric. The reviewer gets the draft and the rubric, not the conversation that produced the draft, so it can't simply agree with reasoning it never saw. Hiding the drafting history covers one kind of shared blind spot. The other kind lives in the model. If the writer is a model that tends to see a loosely related aside as a natural connection, a grader built the same way will tend to see it that way too, and the rubric line meant to catch it passes. So since 2026-08-19 our written rule has been that the reviewer runs on a different model from the main session. The agent file said model: opus , and the rule assumed the main session would be on a model from another family by default. That assumption was the whole mechanism. Nothing checked it. On 2026-09-26 one of our direct-message replies pulled in a measurement of ours that had little to do with what the other person was talking about, and the tone review passed the connection. Around the same time a reviewer, partway through another review, compared its own transcript with the agent file and reported that it was running on the same model as the main session. We put three options to the owner: move the reviewer to a model from another family, move it to a cheaper model, or redefine the rule as "a reviewer that hasn't seen the drafting history" and drop the model requirement. The owner chose a variant of the first: Opus by default, the other family whenever the main session itself is on Opus, and a mechanism instead of a reminder. Our diagnosis at the time was that the opus alias had started pointing at a newer Opus version, the one the main session was already using. While writing this article we reread the subagent docs and found a rule that gives the same result wherever the alias points. In the docs' words, a family alias such as opus "resolves to the main conversation's model instead of the version the alias points to" when "the main conversation's model belongs to that family". Every main session that launched a reviewer from 2026-09-23 on ran on Opus, so our records can't tell the two explanations apart. Both lead to the same conclusion: from an Opus main session, asking for opus cannot get you a different model. For this article we counted. On our machine, Claude Code keeps a transcript for every subagent run in a folder next to the session's own transcript, plus a small .meta.json file that names the agent type and the description the run was launched with. Our oldest surviving transcripts start on 2026-09-13. We matched each deep-reviewer launch to the run it produced, read the model name on the reviewer's own answers, and compared it with the main session's model at the moment of the launch. Before the hook, 190 launches matched a run. In 101, the main session was on another family and the reviewer on Opus, which is what the rule intended. In the other 89, the reviewer was not just in the main session's family but on its exact model: 24 on 2026-09-13 and 09-16, when the main session ran an earlier Opus version, and 65 between 09-23 and 09-27 on claude-opus-5-5. The problem we noticed on 09-26 had already happened on 2026-09-13, the oldest day the transcripts reach back to. Nobody saw it then, because a verdict file looks the same whichever model wrote it. The hook is a shell script, scripts/hooks/require-cross-model-reviewer.sh , registered in .claude/settings.json as a PreToolUse hook with the matcher Agent|Task . It went in on 2026-09-27 at 10:08. Claude Code hands a command hook a JSON object on stdin, and that object includes transcript path , the path to the session's transcript, along with the tool's input. For the Agent tool, the input carries subagent type , description and an optional model . The script lets every launch through unless subagent type is deep-reviewer . For a reviewer launch, it reads the last 400 lines of the main transcript, keeps the assistant entries, skips the ones marked <synthetic , and takes the last model name it finds. Here is the decision, with the Japanese error messages translated and shortened: main model="$ tail -n 400 "$transcript path" | jq -rR 'fromjson? | select .type == "assistant" | .message.model // empty' 2 /dev/null | grep -v '^<synthetic $' | tail -n 1 || true " if -z "$main model" ; then exit 0 fi case "$main model" in opus if "$requested model" = "fable" ; then echo "Main is ${main model} Opus : launch deep-reviewer with model: \"fable\"" &2 exit 2 fi ;; if -n "$requested model" && "$requested model" = "opus" ; then echo "Main is ${main model}: launch deep-reviewer with no model or model: \"opus\"" &2 exit 2 fi ;; esac exit 0 For PreToolUse , exit code 2 blocks the tool call, and Claude receives the stderr text as the reason. In six of the seven real blocks below, the session read the message and relaunched the reviewer with the model it named. When the hook can't read a model from the transcript, it lets the launch through. We chose that on purpose: we didn't want a block that nothing in the session could satisfy. The family test is a substring match on opus . When the main session runs anything else, the hook accepts the reviewer's default opus and blocks any other explicit model. Between the hook going in and 19:00 on 2026-10-13, our sessions made 851 Agent calls. 596 of them launched deep-reviewer , and every one of those came from a main session running on Opus, so the non-Opus branch has only ever run in the test file. The hook blocked 8 launches. The first was ours: about a minute before the commit, the session launched a reviewer with no model on purpose, to see the hook stop it. The other 7 were real: model: "fable" within a minute. model: "opus" . It was relaunched with fable 24 seconds later. general-purpose , still with The hook ignores general-purpose , so that review ran on claude-opus-5-5, the main session's model. The transcript doesn't record why the session made that choice. What it does record is that the session changed the one field the hook keys on, and got a review on the writer's own model. The 66 reviewer launches outside reply review that did get through all ran on the other family, according to their own transcripts. The hook went in at 10:08. At 15:54 the same day, a second commit carved out reply reviews, the reviews we run far more often than any other. The owner's reasoning, paraphrased: tone criteria are vague by nature, so verdicts will drift between runs and that is acceptable; for now every reply review should run on the same Opus model, even when Opus wrote the reply, and each reply should get at least two reviewers instead of one. So the hook has a second branch, checked first. A launch whose description starts with the Japanese for "reply review" must be a deep-reviewer , and its model must be unset or opus , whatever the main session runs. From that commit until 19:00 on 2026-10-13, 526 reply-review runs could be matched to their transcripts, and every one ran on claude-opus-5-5, the same model as the main session that wrote the replies. For the review we run most, writer and grader share a model by decision, not by accident. What replaced model diversity there is a second opinion. Each reply goes to at least two reviewers who don't see each other's verdicts. A rubric criterion both pass is fine, a criterion both fail sends the reply back, and a split criterion is decided by the main session re-checking only that criterion and writing down why. Two reviewers only help if both verdicts really came from reviewers. The main session could write a verdict file itself, or tidy one up afterwards, and the file would look the same. So the same commit added src/marketing/reply-reviewer-provenance.ts , which the tools that count reply verdicts call for every verdict file. For each file it looks through the subagent transcripts that changed after the file did, finds the run that wrote that path with the Write tool, and checks four things: the run's .meta.json says deep-reviewer , its description starts with the reply-review marker, every model name in its transcript is claude-opus-5-5 , and the last content it wrote equals what is on disk now compared as JSON, so whitespace doesn't matter . If any check fails, the verdict isn't counted. If the transcripts can't be found at all, it says so and stops. That is the opposite choice from the hook, which lets the launch through when it can't tell; here a wrong guess only delays a reply. This is the part that sees what the hook can't: the model that actually answered, not the one that was asked for. Our local error log, which isn't committed, has 4 runs where this check refused a verdict: on 09-30, 10-04, 10-05 and 10-13. All four gave the same reason: the content had changed after the reviewer wrote it. We traced each one. In all four, the reviewer itself had written the file with Write and then fixed a detail with Edit . The check reads only Write calls, so the reviewer's own correction looked like someone else's edit. Each time, the main session asked the same reviewer to rewrite the file in a single Write , and the round went through. None of the four was about a wrong model, a wrong agent type, or a file the main session wrote. So far the strictest check we have has caught only honest mistakes, which is what we would see if nothing went wrong, and also what we would see if it were looking in the wrong place. The model that actually runs. The hook checks the requested value. The subagent docs list ways the running model can differ from the request: a mod's spawn hook can replace the per-invocation model, an organization's model allowlist can substitute another one, and the family-alias rule quoted above maps opus onto the main session's model. For reviews other than replies, nothing compares the requested model with the one that answered. The counts in this article are the first time we did, with a one-off script. Other agent types. The 10-12 relaunch is one case. After the hook went in, nine launches, that one included, ran as general-purpose with model: "opus" and had "review" in their description. Whether those should have been on another family is a policy question the hook can't answer, because it has no idea which launches are reviews. A transcript that lags or is missing. The hooks docs say the transcript is written asynchronously and may not yet include the current turn's latest messages when a hook fires. The hook takes the last model it finds, so a model switch earlier in the same turn could leave it judging against the old one. If no model can be read, the launch goes through. Its own failure. We haven't set onFailure on this hook. Per the docs, a command hook that exits with any code other than 0 or 2, or fails to start, is a non-blocking error and the tool call proceeds; "onFailure": "block" changes that. The script runs with set -e , so if jq were missing it would exit with an ordinary error code and every reviewer launch would go through. Whether another family catches more. We have one case, and it doesn't settle anything. The direct-message reply that started all this passed its same-model review on 09-26, and that reviewer's transcript adds a twist: the draft it passed was a rewrite it had suggested itself in an earlier round, which it pointed out in its own report. On 09-27 at 10:58 we re-graded the same text with a reviewer from the other family, on the rubric as it was committed then, and it passed every criterion again. The rubric simply had no line for it. Shortly after, we tested a rubric revision that added two criteria, "answers the other person's point" and "doesn't change the subject", against recorded replies including that one. Two reviewers from the other family flagged it under the new criteria. Later that day, on a slightly revised rubric, two claude-opus-5-5 reviewers still held the reply back, but under an older criterion, and passed it on both new ones. After the subject-change criterion was reworded more concretely, two more claude-opus-5-5 reviewers flagged it there. One reply, four rubric versions, eight reviewers: the clearest effect in it is the rubric's wording, not the model. The middle step fits "same model, same blind spot", and it fits "the wording was still too vague" just as well. PreToolUse hook can also return updatedInput and change the model itself. We block so the session sees the reason, and the 10-12 case shows the cost: a blocked session can pick a route the hook doesn't watch. If you have measured whether a reviewer from another model family catches more than one from the writer's own family, or decided it isn't worth enforcing, we'd like to read how you got there in the comments below. We'll answer each one there.