The Agentic Code Review Stack: Catching 'Almost Right' Bugs with Kiro CLI, GitHub Actions, and Sandboxed Agents

Originally published on tamiz.pro.

Every engineer has shipped an "almost right" fix. The code compiles. The unit tests pass. The PR looks clean. Three weeks later, a production incident trace points straight to the one branch you didn't test — the null-coalescing chain that silently swallowed an empty array, or the race condition that only appears when two jobs run in parallel. These are not logic errors; they are contextual errors, and they are precisely what conventional review tooling misses.

This tutorial builds a complete agentic code review stack that catches "almost right" bugs before they reach main. You'll combine Kiro CLI (AWS's agentic coding CLI) for autonomous analysis, GitHub Actions for orchestration and gating, and sandboxed autonomous agents (Docker-isolated) for safe, repeatable execution. By the end, you'll have a merge-gated pipeline that reviews every PR with an agent that can actually run your code, explore the call graph, and flag the subtle semantic drift that linters and human eyes skip.

Why this stack? Kiro CLI brings agentic reasoning to the terminal; GitHub Actions gives you a reliable trigger and a merge gate; the sandbox gives you the confidence to let an autonomous agent run untrusted code without burning down your infrastructure.

Let's define the target precisely. An "almost right" bug is one that satisfies all static and shallow checks but violates a runtime or contextual invariant. It's not a typo, and it's not a missing semicolon — the compiler is happy, the linter is silent, and the existing test suite is green.

Flavor	Example	Why it slips through
Semantic drift	if (items.length > 0) instead ofif (items?.length)	Type checker can't see the runtime undefined
Concurrency hazard	Shared mutable state across async handlers	Unit tests run single-threaded
Contract violation	Function returns null where the API contract says[]	No integration test exercises the boundary

Human reviewers are bad at these because they review diffs, not executions. A human reads the changed lines; an agentic reviewer can execute the code, trace the data flow, and compare the behavior against the module's own contract.

Linters (ESLint, Ruff, etc.) operate on the AST. They catch patterns, not semantics. TypeScript catches type mismatches, but it can't know that Array.prototype.sort mutates in place and that your caller relied on the original order. These tools are necessary but insufficient.

An autonomous agent with a sandbox can:

null, NaN, huge numbers) to the changed functions. main. That last capability — writing a failing test — is the killer feature. A bug that can be demonstrated by a failing test is a bug that cannot be argued away.

Before we write any code, make sure you have the following.

ubuntu-latest runner with Docker).pull-requests: write and checks: write permissions (for posting comments and creating status checks).

Component	Role
Kiro CLI	The agentic "brain" — parses the diff, reasons about the code, produces structured findings
GitHub Actions	The "trigger and gate" — fires on PR events, runs the pipeline, blocks merge on failure
Sandboxed agent	The "hands" — wraps Kiro in an isolated Docker container with no network, read-only FS, and resource limits
Orchestrator	The "policy layer" — parses Kiro's JSON output, filters by severity, applies rules, sets the status check

Here's the end-to-end flow, as a diagram-in-words:

[PR opened / updated]
        │
        ▼
[GitHub Actions workflow: agentic-review]
        │
        ├── checkout PR branch
        ├── build project (to catch compile errors early)
        ├── pull sandbox image (cached)
        ├── run sandboxed Kiro agent  ──►  [Docker: --network none, --read-only, --memory 2g]
        │                                     │
        │                                     └── writes review-output.json
        ├── run orchestrator (policy layer)
        │       ├── parse findings
        │       ├── filter by severity / rule allowlist
        │       └── emit status: success | failure | neutral
        └── post PR comment + set status check

The pipeline has four distinct layers:

Keeping these layers separate is the key to maintainability. If you want to swap Kiro for another agent (e.g., claude -p or aider), you only touch the cognition layer. If you want to change the merge policy, you only touch the policy layer.

Kiro CLI is the headless, CI-friendly interface to the Kiro agentic environment. It lets you run an autonomous coding agent from the command line, which is exactly what we need for a PR review pipeline.

npm install -g @aws/kiro-cli

curl -fsSL https://kiro.dev/install.sh | bash

kiro --version

Note: Kiro CLI is under active development. Pin a specific version in your CI (npm install -g @aws/kiro-cli@<version>) so your pipeline doesn't break when a new release ships.

Kiro needs credentials to call the underlying model. In CI, use a secret:

export KIRO_API_KEY="${{ secrets.KIRO_API_KEY }}"
kiro auth login --api-key "$KIRO_API_KEY"

For local development, you can use the interactive login:

kiro auth login

Create a kiro.config.json at the repo root (or .kiro/config.json):

{
  "model": "kiro-1.5-pro",
  "mode": "review",
  "maxTokens": 8000,
  "temperature": 0.1,
  "timeoutMs": 120000,
  "review": {
    "scope": "diff",
    "includeTests": true,
    "severityThreshold": "warning",
    "outputFormat": "json"
  }
}

Key fields:

"mode": "review" — tells Kiro to behave as a reviewer, not a code generator."scope": "diff" — limits the agent's attention to the PR diff (critical for cost and latency)."outputFormat": "json" — gives us machine-readable findings for the orchestrator.

kiro run --prompt "Review src/utils/date.ts for edge-case bugs" --output review.json
cat review.json | jq '.findings | length'

If this prints a number (even 0), Kiro is working.

Now we wrap Kiro in a Docker sandbox. The sandbox is what makes this stack safe — it means an autonomous agent can run arbitrary code from a PR without risking your CI infrastructure.

Create sandbox/Dockerfile:

FROM node:20-slim

RUN apt-get update \
    && apt-get install -y --no-install-recommends git jq \
    && rm -rf /var/lib/apt/lists/*

RUN npm install -g @aws/kiro-cli@1.2.3

RUN useradd -m -u 1000 agent
USER agent

WORKDIR /workspace

COPY --chown=agent:agent run-agent.sh /usr/local/bin/run-agent.sh
RUN chmod +x /usr/local/bin/run-agent.sh

ENTRYPOINT ["run-agent.sh"]

Create sandbox/run-agent.sh:

#!/usr/bin/env bash
set -euo pipefail


export KIRO_API_KEY="${KIRO_API_KEY:?KIRO_API_KEY is required}"

cd /workspace/repo

DIFF_SIZE=$(wc -l < /workspace/diff.patch)
PROMPT="You are a senior code reviewer. Review the changes in /workspace/diff.patch.
Focus ONLY on bugs that would pass existing tests but fail in production:
- semantic drift (e.g., null vs empty array)
- concurrency hazards (shared mutable state)
- contract violations (return type vs API contract)
- edge cases (empty inputs, boundary values)

For each finding, provide:
1. A severity (critical | warning | info)
2. A file path and line number
3. A one-sentence explanation
4. A concrete reproduction (a test that would fail)

Output your findings as JSON to /workspace/review-output.json.
Do not modify any source files. Do not run the test suite unless asked.
Diff has ${DIFF_SIZE} lines."

kiro run \
  --prompt "$PROMPT" \
  --config /workspace/kiro.config.json \
  --output /workspace/review-output.json

if ! jq -e '.findings' /workspace/review-output.json > /dev/null 2>&1; then
  echo "ERROR: Kiro did not produce valid findings JSON" >&2
  exit 1
fi

echo "Review complete: $(jq '.findings | length' /workspace/review-output.json) findings"
docker build -t kiro-sandbox:latest ./sandbox

Why a separate Dockerfile? You could run Kiro directly on the runner, but then the agent has access to your CI secrets, your network, and your entire filesystem. The sandbox is the difference between "agentic review" and "agentic liability."

Now we create the GitHub Actions workflow that triggers on every PR and runs the sandboxed agent.

Create .github/workflows/agentic-review.yml:

name: Agentic Code Review

on:
  pull_request:
    types: [opened, synchronize, reopened]

permissions:
  contents: read
  pull-requests: write
  checks: write

jobs:
  agentic-review:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - name: Checkout PR branch
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Node
        uses: actions/setup-node@v4
        with:
          node-version: '20'

      - name: Install dependencies
        run: npm ci

      - name: Build project (early compile check)
        run: npm run build
        continue-on-error: true

      - name: Generate PR diff
        run: |
          git diff origin/${{ github.base_ref }}...HEAD > diff.patch
          echo "Diff lines: $(wc -l < diff.patch)"

      - name: Build sandbox image
        run: docker build -t kiro-sandbox:latest ./sandbox

      - name: Run sandboxed Kiro agent
        env:
          KIRO_API_KEY: ${{ secrets.KIRO_API_KEY }}
        run: |
          docker run --rm \
            --name kiro-review \
            --network none \
            --memory 2g \
            --cpus 2 \
            --read-only \
            --tmpfs /tmp \
            -v "$PWD/diff.patch:/workspace/diff.patch:ro" \
            -v "$PWD:/workspace/repo:ro" \
            -v "$PWD/kiro.config.json:/workspace/kiro.config.json:ro" \
            -v "$PWD/review-output.json:/workspace/review-output.json" \
            -e KIRO_API_KEY="$KIRO_API_KEY" \
            kiro-sandbox:latest
        continue-on-error: true

      - name: Run orchestrator
        run: node orchestrator.mjs

      - name: Post review comment
        if: always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const findings = JSON.parse(fs.readFileSync('review-output.json', 'utf8'));
            const summary = findings.findings || [];
            const critical = summary.filter(f => f.severity === 'critical');
            const warning = summary.filter(f => f.severity === 'warning');

            let body = `## 🤖 Agentic Review\n\n`;
            body += `**Findings:** ${summary.length} (${critical.length} critical, ${warning.length} warning)\n\n`;
            if (summary.length === 0) {
              body += `No issues found. ✅\n`;
            } else {
              body += `| Severity | File | Line | Finding |\n|---|---|---|---|\n`;
              for (const f of summary.slice(0, 20)) {
                body += `| ${f.severity} | \`${f.file}\` | ${f.line} | ${f.message} |\n`;
              }
            }

            await github.rest.issues.createComment({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number,
              body: body
            });

Key details:

--network none — the agent cannot exfiltrate data or call external APIs.--read-only — the agent cannot modify the repo (it only writes to the mounted review-output.json).--memory 2g --cpus 2 — caps resource usage so a runaway agent can't kill the runner.continue-on-error: true — if Kiro crashes, we still run the orchestrator so the PR gets an "inconclusive" comment instead of a silent failure. The orchestrator is the policy brain. It reads Kiro's JSON output, filters findings according to your rules, and emits a status check that GitHub branch protection can enforce.

Create orchestrator.mjs:

import fs from 'node:fs';
import path from 'node:path';

const REVIEW_OUTPUT = 'review-output.json';

// ---- Policy configuration ----
const POLICY = {
  // Severities that block the merge
  blocking: ['critical'],
  // Severities that are reported but do not block
  reportOnly: ['warning', 'info'],
  // Files to ignore (e.g., generated code, lockfiles)
  ignorePaths: [/\.lock$/, /dist\//, /node_modules\//],
  // Max findings to report before we consider the review "flooded"
  maxFindings: 50,
  // If Kiro crashed (no valid output), treat as inconclusive, not failure
  inconclusiveOnError: true,
};

function loadFindings() {
  if (!fs.existsSync(REVIEW_OUTPUT)) {
    return { findings: [], error: 'No review output produced' };
  }
  try {
    const raw = fs.readFileSync(REVIEW_OUTPUT, 'utf8');
    const parsed = JSON.parse(raw);
    return { findings: parsed.findings || [], error: null };
  } catch (e) {
    return { findings: [], error: `Failed to parse: ${e.message}` };
  }
}

function applyPolicy(findings) {
  return findings
    .filter((f) => !POLICY.ignorePaths.some((re) => re.test(f.file || '')))
    .slice(0, POLICY.maxFindings);
}

function classify(findings) {
  const blockers = findings.filter((f) => POLICY.blocking.includes(f.severity));
  const reports = findings.filter((f) => POLICY.reportOnly.includes(f.severity));
  return { blockers, reports };
}

function setStatusCheck(conclusion, summary) {
  // In GitHub Actions, write the status to a file that the workflow reads
  const payload = {
    conclusion,
    summary,
    blockers: summary.blockers,
    reports: summary.reports,
  };
  fs.writeFileSync('status-check.json', JSON.stringify(payload, null, 2));
  console.log(`Status: ${conclusion}`);
  console.log(`Blockers: ${summary.blockers.length}, Reports: ${summary.reports.length}`);
}

// ---- Main ----
const { findings, error } = loadFindings();

if (error && POLICY.inconclusiveOnError) {
  setStatusCheck('neutral', {
    blockers: [],
    reports: [],
    error,
    message: 'Agentic review was inconclusive — falling back to human review.',
  });
  process.exit(0);
}

const filtered = applyPolicy(findings);
const { blockers, reports } = classify(filtered);

if (blockers.length > 0) {
  setStatusCheck('failure', { blockers, reports });
  process.exit(1); // This makes the GitHub Actions job fail → blocks merge
} else {
  setStatusCheck('success', { blockers, reports });
  process.exit(0);
}

blocking: ['critical']`` ignorePaths``inconclusiveOnError`` neutral rather than failure. A broken agent should not block a good PR. The docker run command in the workflow is already fairly locked down, but for production you'll want to go further. Here's a hardened version:

docker run --rm \
  --name kiro-review \
  --network none \
  --memory 2g \
  --memory-swap 2g \
  --cpus 2 \
  --pids-limit 256 \
  --read-only \
  --tmpfs /tmp:size=64m \
  --tmpfs /run:size=16m \
  --cap-drop ALL \
  --security-opt no-new-privileges \
  --security-opt seccomp=default \
  -v "$PWD/diff.patch:/workspace/diff.patch:ro" \
  -v "$PWD:/workspace/repo:ro" \
  -v "$PWD/kiro.config.json:/workspace/kiro.config.json:ro" \
  -v "$PWD/review-output.json:/workspace/review-output.json" \
  -e KIRO_API_KEY="$KIRO_API_KEY" \
  kiro-sandbox:latest

Flag	Purpose
--network none	No network egress — the agent can't call home or exfiltrate secrets
--cap-drop ALL	Drops all Linux capabilities — no raw sockets, no ptrace, no mount
--security-opt no-new-privileges	Prevents the agent from escalating privileges via setuid binaries
--pids-limit 256	Stops fork bombs
--read-only	The root FS is read-only; the agent can only write to the mounted output file
--tmpfs	Ephemeral writable space that vanishes on container exit

Important: Never mount the CI secrets directory into the sandbox. Pass only the API key via environment variable, and even then, consider using a short-lived, scoped token that Kiro can't reuse after the job ends.

Now we make the review mandatory. GitHub branch protection rules are the enforcement mechanism.

The orchestrator writes status-check.json, but GitHub Actions needs a real check. The cleanest way is to use the actions/github-script step to create a check run, or simply let the job's exit code drive a required status check.

Add this to the workflow after the orchestrator step:

- name: Create status check
  if: always()
  run: |
    node -e "
      const fs = require('fs');
      const status = JSON.parse(fs.readFileSync('status-check.json', 'utf8'));
      // The job itself already exits 1 on blockers, so GitHub marks it failed
      console.log('Conclusion:', status.conclusion);
    "

Actually, the simplest robust approach: the orchestrator's process.exit(1) makes the GitHub Actions job fail. Then in branch protection, require the agentic-review check to pass.

In GitHub → Settings → Branches → Add rule:

Agentic Code Review check (or the job name agentic-review). Now, if the agent finds a critical bug, the check fails, and the merge button is disabled.

Kiro's "scope": "diff" is your friend. Reviewing the entire repo on every PR is slow and expensive. For a 500-line PR, the diff is the only thing that matters.

Cap tokens and time:

{
  "review": {
    "maxTokens": 8000,
    "timeoutMs": 120000,
    "maxFiles": 20,
    "maxDiffLines": 1000
  }
}

If the diff exceeds the budget, the agent should review a sample and mark the review as "partial" — better than timing out.

Building the Docker image on every PR is wasteful. Use docker/build-push-action with a cache, or push to a registry and pull it in CI:

- name: Pull sandbox image
  run: docker pull ghcr.io/your-org/kiro-sandbox:latest

Agentic review should augment human review, not replace it. Configure the orchestrator so that if a finding touches a file in CODEOWNERS with a critical path (e.g., auth, payments), it always escalates to a human, even if the agent says it's fine.

Log every finding to a metrics store. After a month, compute your false-positive rate (findings that a human dismissed). If it's above 30%, tighten the prompt or raise severityThreshold.

Scenario	Desired behavior	Implementation
Kiro times out	Mark check as neutral , don't block	timeout-minutes +inconclusiveOnError: true
Sandbox OOM	Retry once with lower memory, then neutral	--memory + retry logic in workflow
Malicious PR tries to escape sandbox	Container is killed, job fails, no host compromise	--cap-drop ALL ,--network none ,--pids-limit
Agent hallucinates a finding	Human dismisses it; we track it	False-positive metrics + allowlist
Huge diff (>1000 lines)	Review a sample, mark as partial	maxDiffLines budget
Kiro CLI not on runner	Fall back to human-only review	continue-on-error +neutral check
Secrets in repo	Sandbox has no network, so exfiltration is impossible	--network none

The worst case is a PR that contains a Dockerfile or a script designed to break out of the container. The --cap-drop ALL, --security-opt no-new-privileges, and --network none flags make this very hard. But you should also ensure the host runner is itself ephemeral (GitHub-hosted runners are) so that even a successful escape only compromises a throwaway machine.

No. It replaces the boring 80% — the mechanical "did you handle the empty array" checks. The human should focus on architecture, design intent, and API contracts. Configure branch protection to require both the agentic check and a human approval.

Three strategies:

critical finding. If the agent can't produce one, the finding is downgraded.

source & further reading

dev.to — original article ForgeInfra- We Built an AI That Repairs Legacy Infrastructure Software at HackUTD 2026 Why LLMs cannot handle legal deadlines (and how to fix it with MCP) My Independent Open-Source Protocol Paper ("DoSync") Was Accepted at IEEE CCNC 2027