# 🌟 The CI Gate Rejected the Terraform Change—but the LLM Still Ran

> Source: <https://dev.to/pravesh_sudha_3c2b0c2b5e0/the-ci-gate-rejected-the-terraform-change-but-the-llm-still-ran-3hfg>
> Published: 2026-09-02 16:57:13+00:00

How I added a deterministic trace contract to an AI Terraform reviewer so rejected infrastructure changes stop before the model is invoked.

I recently built an [AI-powered Terraform review agent](https://github.com/Pravesh-Sudha/ai-devops-agent/tree/main/terraform-review-agent) that combines Terrascan, GitHub Actions, AWS Lambda and Gemini.

The workflow is straightforward:

The project worked. A risky change produced `REJECT`

, and GitHub Actions failed the pull request as expected.

But while reviewing the execution path, I noticed an important problem: **the correct final verdict did not prove that the control was enforced at the correct boundary.**

The pipeline rejected the change—but the LLM had already run.

That distinction matters in DevOps. A policy that says “do not continue” should stop the next action. It should not merely ask the next action to agree that the request should have been stopped.

This article shows how I used AgentInspect to make that path visible and add a deterministic CI contract around it.

Disclosure:I tested AgentInspect independently in the workflow described here. The maintainer reviewed the AgentInspect commands for technical accuracy; the conclusions are my own. AgentInspect did not replace Terrascan, GitHub Actions, AWS controls or the application’s security policy.

My Terraform review project uses a practical serverless workflow:

```
Terraform pull request
        ↓
GitHub Actions
        ↓
Terrascan JSON report
        ↓
AWS Lambda
        ↓
Gemini review
        ↓
APPROVE | APPROVE_WITH_CHANGES | REJECT
```

The Lambda function extracts relevant Terrascan findings and sends a bounded structure to Gemini. The prompt contains explicit decision rules:

The [GitHub Actions workflow](https://github.com/Pravesh-Sudha/ai-devops-agent/blob/main/.github/workflows/main.yml) then reads the returned verdict and exits with status `1`

when the model returns `REJECT`

.

At first, this looked like a security gate. The result was correct and the PR was blocked.

The problem was where the authority lived.

In the original [Lambda implementation](https://github.com/Pravesh-Sudha/ai-devops-agent/blob/main/terraform-review-agent/lambda/lambda_function.py), the sequence was effectively:

```
findings = extract_relevant_findings(results)
prompt = build_prompt(findings)

ai_review = call_gemini(prompt)
verdict = extract_verdict(ai_review)
```

The risk thresholds were described inside the prompt. They were not evaluated as a deterministic control before the provider call.

This creates three different concerns:

The third concern is the easiest one to miss in ordinary CI output.

Imagine a fixture containing a HIGH-severity public-access violation. A conventional test might assert:

```
expect(result.verdict).toBe("REJECT");
```

That assertion passes whether the pipeline rejects before Gemini or calls Gemini and then accepts its rejection.

Those paths are not equivalent:

```
Desired
Terrascan → deterministic policy → REJECT → stop

Original
Terrascan → Gemini → parse response → REJECT → stop
```

The final value is the same. The control boundary is different.

For the blocked test case, I wanted to assert a stronger invariant:

Terrascan and the deterministic policy check must execute, and the LLM path must execute zero times.

That is an execution contract, not an answer-quality evaluation.

The existing Lambda is written in Python, while AgentInspect is a TypeScript-first toolkit. I did not pretend that it could automatically instrument the Python function.

Instead, I added a small Node.js evidence runner at the CI boundary. It wraps the operations the pipeline owns: reading the scan, evaluating the policy and, only when appropriate, invoking the existing Lambda.

For this test, I used AgentInspect `6.17.4`

and pinned the version so the CI behavior would not move underneath the experiment:

```
npm install agent-inspect@6.17.4
```

The simplified runner looks like this:

``` js
import { inspectRun, step } from "agent-inspect";

const traceDir = ".agent-inspect/terraform-review";

function evaluatePolicy(report) {
  const violations = report.violations ?? [];
  const severities = violations.map((item) =>
    String(item.severity ?? "").toUpperCase()
  );

  const highOrCritical = severities.some(
    (severity) => severity === "HIGH" || severity === "CRITICAL"
  );
  const mediumCount = severities.filter(
    (severity) => severity === "MEDIUM"
  ).length;

  if (highOrCritical || mediumCount >= 4) {
    return { verdict: "REJECT", reason: "risk-threshold" };
  }

  if (mediumCount >= 1) {
    return {
      verdict: "APPROVE_WITH_CHANGES",
      reason: "medium-findings"
    };
  }

  return { verdict: "APPROVE", reason: "low-or-info-only" };
}

const result = await inspectRun(
  "terraform-ai-review",
  async () => {
    const report = await step.tool("terrascan", () =>
      readTerrascanReport("terrascan_report.json")
    );

    const policy = await step.tool("evaluate_policy", () =>
      evaluatePolicy(report)
    );

    if (policy.verdict === "REJECT") {
      return policy;
    }

    const review = await step.llm("gemini-2.5-flash", () =>
      invokeTerraformReviewLambda(report)
    );

    return {
      ...review,
      verdict: policy.verdict
    };
  },
  {
    traceDir,
    silent: process.env.CI === "true",
    metadata: {
      workflow: "terraform-ai-review",
      fixture: "high-severity-public-access"
    }
  }
);

console.log(result);
```

This example deliberately keeps the policy small. The HTTPS rule needs its own structured input or a stable mapping from specific Terrascan findings. I would not implement it by searching arbitrary free text and call that deterministic.

The important change is architectural: **code owns the risk threshold and final CI authority; the model can provide explanation and remediation only after the deterministic gate allows that path.**

I ran a controlled fixture representing a HIGH-severity finding and inspected the local trace:

```
npx agent-inspect list --dir .agent-inspect/terraform-review
npx agent-inspect view <run-id> --dir .agent-inspect/terraform-review
```

Before the short-circuit, the execution contained an LLM step:

```
terraform-ai-review
├─ tool:terrascan              success
└─ llm:gemini-2.5-flash       success
```

After moving policy enforcement ahead of the provider call, the rejected path became:

```
terraform-ai-review
├─ tool:terrascan              success
└─ tool:evaluate_policy        success
```

The absence of the LLM step was now visible, but I did not want reviewers to verify it manually on every pull request. The next step was turning it into a deterministic check.

AgentInspect supports deterministic checks over retained local traces. For this case, I used a JSON check configuration:

```
{
  "checks": {
    "tool": {
      "required": ["terrascan", "evaluate_policy"]
    },
    "llm": {
      "maxCalls": 0
    }
  }
}
```

Then CI checks the trace produced by the blocked fixture:

```
npx agent-inspect check .agent-inspect/terraform-review \
  --config blocked-path.check.json \
  --require-completed \
  --json
```

The broken path fails because the model call count is greater than zero. The fixed path passes because:

`terrascan`

appeared.`evaluate_policy`

appeared.This is the exact behavior I wanted from CI. It does not ask another model whether the trace looks safe. It evaluates a small, reproducible structural contract and returns a deterministic exit code.

A CI failure without usable evidence usually starts another debugging cycle. To retain a bounded report, I added an artifact step:

```
- name: Run blocked-path regression
  run: node scripts/run-blocked-policy-case.mjs

- name: Check blocked-path contract
  run: |
    npx agent-inspect check .agent-inspect/terraform-review \
      --config blocked-path.check.json \
      --require-completed \
      --json > trace-contract-result.json

- name: Build safe trace artifacts
  if: always()
  run: |
    npx agent-inspect artifacts .agent-inspect/terraform-review \
      --output-dir ./agent-inspect-artifacts \
      --github-summary "$GITHUB_STEP_SUMMARY"

- name: Upload review evidence
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: terraform-agent-trace
    path: |
      agent-inspect-artifacts/
      trace-contract-result.json
    retention-days: 14
```

AgentInspect creates the local report; GitHub Actions owns the upload and retention. For real pipeline data, I would still review the exact derived artifact before sharing it outside the repository. Redaction and safety scans are safeguards, not compliance certification.

My first implementation treated the final verdict as proof that the gate worked. It was only proof that the workflow ended with the expected string.

The stronger DevOps questions are:

This is where execution traces are useful. They provide evidence about the path rather than only the answer.

There is also a useful separation of responsibilities:

| Layer | Responsibility |
|---|---|
| Terrascan | Detect infrastructure findings |
| Deterministic policy code | Enforce explicit risk thresholds |
| Gemini | Explain findings and suggest remediation on allowed paths |
| AgentInspect | Record and check the execution path |
| GitHub Actions | Enforce the build result and retain reviewed artifacts |

AgentInspect did not make the Terraform deployment secure. It helped me verify whether my own control flow matched the policy I claimed to enforce.

A green trace contract is narrow evidence. It does not prove that every Terraform rule is correct or that the deployed infrastructure is safe.

This workflow still needs:

It also does not prove that Gemini’s remediation advice is good. That requires separate evaluation.

The contract proves one specific invariant: **when deterministic policy rejects a Terraform change, the model path does not run.**

AI can add useful context to DevOps workflows, but it should not own controls that can be expressed clearly in code.

My Terraform review agent already produced the expected rejection. AgentInspect showed me that the route to that answer was weaker than the answer itself suggested.

Moving the risk threshold ahead of the LLM call gave the workflow a cleaner authority boundary. Adding a trace contract made that boundary reviewable in CI.

The lesson is simple:

Do not test only what your agent returned. Test which actions it was allowed to take before returning it.