# CI gates for AI-generated PRs need re-derivable evidence

> Source: <https://dev.to/sjh9714/ci-gates-for-ai-generated-prs-need-re-derivable-evidence-2g4e>
> Published: 2026-06-21 01:08:59+00:00

When a CI gate flags an AI-generated PR, the important question is not only "what did it flag?"

It is also:

"Could someone else come back later and re-derive why this finding fired?"

That is the reason I added evidence snapshots to Agent Gate v0.2.1.

Agent Gate is a GitHub Action for AI-generated pull requests.

It does not review code with an LLM. It checks deterministic merge evidence in CI:

The Action does not checkout PR code, call LLMs at runtime, or execute repository scripts.

In v0.2.0, Agent Gate added stable finding IDs.

That gave every finding a short audit handle, for example:

```
agf_987ab9ddb8c1b299
```

That is useful for references, comments, future override workflows, and log-based debugging.

But an ID by itself is not proof. If someone sees the ID later, they still need to know what recorded material produced it.

v0.2.1 adds `evidenceSnapshot`

to public findings.

The split is:

```
findingId = short audit handle
evidenceSnapshot = canonical material used to derive that handle
```

The snapshot is intentionally boring. It contains stable rule material such as:

It does not include timestamps, report order, risk score, version, commit SHA, or mutable display text.

Example compact log output:

```
Agent Gate: NEEDS HUMAN DECISION
Decision: warn
Risk score: 49 / 100
Why: Agent-generated PRs must include an agent-gate contract.
Recommended next step: Add a PR contract before relying on scope checks.
Policy status: warning today; eligible to become a merge gate after tuning.

Findings:
- error agf_be0c2c2a66312aff contract/missing
- error agf_987ab9ddb8c1b299 risk/high-risk-path .github/workflows/agent-gate.yml
- warn agf_6016e753491255d7 workflow/dangerous-pattern .github/workflows/agent-gate.yml
```

The compact log stays short, but the JSON and Markdown reports carry the fuller evidence.

Example JSON shape:

```
{
  "findingId": "agf_987ab9ddb8c1b299",
  "ruleId": "risk/high-risk-path",
  "severity": "error",
  "path": ".github/workflows/agent-gate.yml",
  "evidenceSnapshot": {
    "ruleId": "risk/high-risk-path",
    "severity": "error",
    "path": ".github/workflows/agent-gate.yml",
    "evidence": [
      {
        "label": "changed_file",
        "value": ".github/workflows/agent-gate.yml"
      }
    ]
  }
}
```

For me, the bar for promoting a finding from warning to blocking is:

```
A third party should be able to re-derive the finding from recorded evidence.
```

That does not mean the check is magically correct.

It means the failure mode is visible, reproducible, and tunable.

A repo can start in warn mode, observe which findings are useful, and only later promote low-noise findings into merge gates.

Agent Gate still does not prove semantic correctness.

Matching test-file evidence is not proof that the tests cover the behavior. It is change evidence / self-consistency evidence.

Maintainer override storage is also not implemented yet. That is probably the next hard design question: if someone bypasses a finding, where should that override live so it is durable enough to inspect later?

CODEOWNERS / reviewer evidence and package dependency drift are also future work.

If you maintain a repo where coding agents open PRs, I would love feedback on whether this kind of evidence is useful or too noisy in observe mode.

Repo:

[https://github.com/sjh9714/Agent-Gate](https://github.com/sjh9714/Agent-Gate)

Disclosure: I maintain Agent Gate. v0.2.1 is still a prerelease; I would start in warn mode before treating any finding as a merge gate.