The core of OpenAI's agentic software factory relies on a risk classifier to decide if a pull request is "low risk." If the system marks it as such and specialist reviewers find no blockers, the change bypasses human review and heads straight to production. While each agent might be doing its job, the real question is whether the system is actually correct in deciding that a human is no longer needed for that specific release.
CI and risk classification are not the same thing #
A green CI run proves the code builds and passes the specific tests and linters in the pipeline. Specialist review agents might also sign off on their specific domains. However, none of that proves the change actually belongs in the low-risk category.
The risk classifier is making a separate claim: that the surface area affected and the expected consequences are safe enough to justify less human scrutiny. If we only monitor the code and the deployment agents, we are ignoring the routing decision itself. The act of removing a human from the loop is a behavior that can be wrong, even if the individual components of the pipeline are functioning as intended.
Risk is contextual and not a permanent label #
Risk isn't a static property of a file or a type of change. Something that was low risk yesterday can become high risk today because a dependency changed, a permission boundary shifted, or a feature suddenly gained a massive user base.
The code can still pass every local check and CI test, but the decision to route it as "low risk" could be wrong because the surrounding environment changed. To actually evaluate this gate, you can't just look at successful deployments. You have to look at the edge cases where the classification might fail, such as:
- A minor code change hitting a component that now has a much larger blast radius.
- A test suite that is still enforcing an outdated constraint after product requirements shifted.
- Deployment agents monitoring a healthy proxy metric while the actual production state is quietly diverging.
- Specialist reviewers all passing a change because the failure happens in the gap between their domains rather than inside one.
Scoring the decision based on outcomes #
The risk gate shouldn't be judged by whether a pull request was green when it arrived at the classifier. Instead, it needs to be judged by what happens after the decision is made.
The only way to validate the "low risk" label is to check if the change actually preserved the required production state. If a supposedly low-risk change triggers a rollback, an incident, or requires manual intervention, then the classification was wrong. The system needs to be evaluated against the consequences of the route it selected, not just the status of the PR.
Next Your AI tests are passing but they might be lying to you →
All Replies （3） #
Finally someone gets it. I'm curious if this works for 5% samples or if you need more to avoid a type 2 error.
Ugh, this burned me last week. My prompts suddenly died after some stealth update. Anyone tried using LangSmith to track this?
I want to try this tonight. I wonder if this works with GitHub Actions or if they're using a custom tool?