Every bot that reads your pull requests is executing code it did not write. Here is how to make sure that is not the end of the story.
There is a quiet assumption baked into most developer tooling: that the code we run is code we trust. Your CI runs your tests. Your linter loads your config. Your build installs your dependencies. For years that held, because the person triggering the job and the person who wrote the code were the same person.
That assumption is now false, and a lot of teams have not noticed.
The moment you add an AI reviewer, a CI job that runs on pull requests from forks, a dependency bot, or any automation that touches contributor code, you have invited untrusted code into a trusted environment. A stranger can now open a pull request and, if you are not careful, get your infrastructure to run something on their behalf. The unsettling part is not the code in the diff. It is everything that runs around it.
This is a field guide to doing that safely. It is written for anyone building or operating a tool that reads other people's repositories, but the same principles apply to any CI pipeline that accepts pull requests from people you do not know.
When people picture untrusted code, they imagine a malicious function hidden in a pull request. That is real, but it is the least of your problems, because you are probably not executing the diff line by line. The real surface is everything your pipeline does automatically on the way to looking at that diff.
Walk through what a normal review or CI job touches:
npm install
, pip install
, or the equivalent executes lifecycle scripts. A single postinstall
hook, or one hostile transitive dependency, runs arbitrary code before a human has looked at anything.npm test
on a contributor's branch, you are running whatever that contributor decided test
should mean today.Here is the sharp version. The attacker does not need you to run their function. They need you to run your tool against their configuration. That is a far wider door, and it is usually left unlocked.
A code execution bug inside an isolated sandbox is a contained incident. The same bug becomes a company-ending event when the environment running untrusted code also holds the keys to everything else.
This is the anatomy of the worst tooling breaches. An automation platform runs contributor code in an environment that, for convenience, also contains its cloud credentials, its database connection string, and, in the most painful cases, the private key of the integration that grants it access to every customer repository. One escape from the sandbox, and the blast radius is not one job. It is every tenant at once.
The lesson is not "sandbox harder." Sandboxes have escapes; assume yours will. The lesson is that the thing reading untrusted code should have nothing worth stealing and nowhere to pivot. If your isolation fails and the intruder finds an empty room with a dead network, you have had an incident worth a postmortem. If they find your master keys, you have had an extinction event. Design so that the worst realistic day is the boring one.
Here is the posture I would defend in a design review. None of it is exotic. It is the discipline of assuming your isolation will eventually fail and making that failure uneventful.
I care about this because I build a tool squarely in this category. ShipGuarde is an AI reviewer that reads pull requests and QAs the running app, which means it reads untrusted code every hour of every day. The posture above is not aspirational for us; it is the architecture. Each pull request is analyzed in a single-use sandbox that holds none of our platform's secrets. The only credential it ever receives is a short-lived token scoped to that one repository, read-only, and it has no route to internal services or metadata. We do not run a repository's own scripts, and we would rather parse a config than obey it. If that sandbox were fully compromised, the intruder would find one repository's readable source and a network cable that goes nowhere. That is the entire point.
I mention it not to sell you anything, but because the constraint shaped every early decision we made, and I think far more tools should be built this way. If you are evaluating any automation that touches your source, this is the posture to make it prove.
If you take nothing else, take this. Before you let any pipeline run on untrusted code, confirm every line: Run down that list honestly. If you cannot check every box, you do not have a hardening problem. You have an untrusted-code problem in a hardening costume.
The industry spent a decade tuning pipelines for speed and convenience on the assumption that everything flowing through them was trusted. AI tooling quietly broke that assumption, and the tools built next will be judged on whether they noticed. The good news is that the safe design is not the slow one. When you stop running other people's code and refuse to hold secrets you do not need, you end up with a system that is both harder to breach and easier to reason about. For once, the secure path and the fast path are the same path. Take it.
Written from the trenches of building an AI reviewer. If your team is working through any of this, I am always happy to compare notes.