The Agentic Code Review Stack: Catching 'Almost Right' Bugs with Kiro CLI, GitHub Actions, and Sandboxed Agents

A developer published a walkthrough of an agentic code review stack that combines Kiro CLI, AWS's agentic coding CLI, with GitHub Actions and Docker-sandboxed autonomous agents to catch "almost right" bugs that pass compilation, linting and unit tests. The pipeline runs the agent in an isolated container with no network and a read-only filesystem, lets it execute changed code and probe edge cases, and gates merges on the resulting findings.

Originally published on tamiz.pro https://tamiz.pro/insights/agentic-code-review-stack-kiro-cli-github-actions-sandboxed-agents . Every engineer has shipped an "almost right" fix. The code compiles. The unit tests pass. The PR looks clean. Three weeks later, a production incident trace points straight to the one branch you didn't test — the null-coalescing chain that silently swallowed an empty array, or the race condition that only appears when two jobs run in parallel. These are not logic errors; they are contextual errors, and they are precisely what conventional review tooling misses. This tutorial builds a complete agentic code review stack that catches "almost right" bugs before they reach main . You'll combine Kiro CLI AWS's agentic coding CLI for autonomous analysis, GitHub Actions for orchestration and gating, and sandboxed autonomous agents Docker-isolated for safe, repeatable execution. By the end, you'll have a merge-gated pipeline that reviews every PR with an agent that can actually run your code, explore the call graph, and flag the subtle semantic drift that linters and human eyes skip. Why this stack? Kiro CLI brings agentic reasoning to the terminal; GitHub Actions gives you a reliable trigger and a merge gate; the sandbox gives you the confidence to let an autonomous agent run untrusted code without burning down your infrastructure. Let's define the target precisely. An "almost right" bug is one that satisfies all static and shallow checks but violates a runtime or contextual invariant. It's not a typo, and it's not a missing semicolon — the compiler is happy, the linter is silent, and the existing test suite is green. | Flavor | Example | Why it slips through | |---|---|---| | Semantic drift | if items.length 0 instead of if items?.length | Type checker can't see the runtime undefined | | Concurrency hazard | Shared mutable state across async handlers | Unit tests run single-threaded | | Contract violation | Function returns null where the API contract says | No integration test exercises the boundary | Human reviewers are bad at these because they review diffs , not executions . A human reads the changed lines; an agentic reviewer can execute the code, trace the data flow, and compare the behavior against the module's own contract. Linters ESLint, Ruff, etc. operate on the AST. They catch patterns , not semantics . TypeScript catches type mismatches, but it can't know that Array.prototype.sort mutates in place and that your caller relied on the original order. These tools are necessary but insufficient. An autonomous agent with a sandbox can: null , NaN , huge numbers to the changed functions. main . That last capability — writing a failing test — is the killer feature. A bug that can be demonstrated by a failing test is a bug that cannot be argued away. Before we write any code, make sure you have the following. ubuntu-latest runner with Docker . pull-requests: write and checks: write permissions for posting comments and creating status checks . | Component | Role | |---|---| | Kiro CLI | The agentic "brain" — parses the diff, reasons about the code, produces structured findings | | GitHub Actions | The "trigger and gate" — fires on PR events, runs the pipeline, blocks merge on failure | | Sandboxed agent | The "hands" — wraps Kiro in an isolated Docker container with no network, read-only FS, and resource limits | | Orchestrator | The "policy layer" — parses Kiro's JSON output, filters by severity, applies rules, sets the status check | Here's the end-to-end flow, as a diagram-in-words: PR opened / updated │ ▼ GitHub Actions workflow: agentic-review │ ├── checkout PR branch ├── build project to catch compile errors early ├── pull sandbox image cached ├── run sandboxed Kiro agent ──► Docker: --network none, --read-only, --memory 2g │ │ │ └── writes review-output.json ├── run orchestrator policy layer │ ├── parse findings │ ├── filter by severity / rule allowlist │ └── emit status: success | failure | neutral └── post PR comment + set status check The pipeline has four distinct layers: Keeping these layers separate is the key to maintainability. If you want to swap Kiro for another agent e.g., claude -p or aider , you only touch the cognition layer. If you want to change the merge policy, you only touch the policy layer. Kiro CLI is the headless, CI-friendly interface to the Kiro agentic environment. It lets you run an autonomous coding agent from the command line, which is exactly what we need for a PR review pipeline. Install globally via npm recommended for CI npm install -g @aws/kiro-cli Or via the official installer script curl -fsSL https://kiro.dev/install.sh | bash Verify kiro --version Note: Kiro CLI is under active development. Pin a specific version in your CI npm install -g @aws/kiro-cli@<version so your pipeline doesn't break when a new release ships. Kiro needs credentials to call the underlying model. In CI, use a secret: export KIRO API KEY="${{ secrets.KIRO API KEY }}" kiro auth login --api-key "$KIRO API KEY" For local development, you can use the interactive login: kiro auth login Create a kiro.config.json at the repo root or .kiro/config.json : { "model": "kiro-1.5-pro", "mode": "review", "maxTokens": 8000, "temperature": 0.1, "timeoutMs": 120000, "review": { "scope": "diff", "includeTests": true, "severityThreshold": "warning", "outputFormat": "json" } } Key fields: "mode": "review" — tells Kiro to behave as a reviewer, not a code generator. "scope": "diff" — limits the agent's attention to the PR diff critical for cost and latency . "outputFormat": "json" — gives us machine-readable findings for the orchestrator. Run Kiro against a single file to confirm it works kiro run --prompt "Review src/utils/date.ts for edge-case bugs" --output review.json cat review.json | jq '.findings | length' If this prints a number even 0 , Kiro is working. Now we wrap Kiro in a Docker sandbox. The sandbox is what makes this stack safe — it means an autonomous agent can run arbitrary code from a PR without risking your CI infrastructure. Create sandbox/Dockerfile : FROM node:20-slim Minimal tooling: git for diffing, jq for JSON parsing RUN apt-get update \ && apt-get install -y --no-install-recommends git jq \ && rm -rf /var/lib/apt/lists/ Install Kiro CLI inside the sandbox RUN npm install -g @aws/kiro-cli@1.2.3 The agent will run as a non-root user RUN useradd -m -u 1000 agent USER agent WORKDIR /workspace Default entrypoint: the agent runner script COPY --chown=agent:agent run-agent.sh /usr/local/bin/run-agent.sh RUN chmod +x /usr/local/bin/run-agent.sh ENTRYPOINT "run-agent.sh" Create sandbox/run-agent.sh : bash /usr/bin/env bash set -euo pipefail The PR diff is mounted at /workspace/diff.patch The repo is mounted read-only at /workspace/repo export KIRO API KEY="${KIRO API KEY:?KIRO API KEY is required}" cd /workspace/repo Generate the prompt from the diff DIFF SIZE=$ wc -l < /workspace/diff.patch PROMPT="You are a senior code reviewer. Review the changes in /workspace/diff.patch. Focus ONLY on bugs that would pass existing tests but fail in production: - semantic drift e.g., null vs empty array - concurrency hazards shared mutable state - contract violations return type vs API contract - edge cases empty inputs, boundary values For each finding, provide: 1. A severity critical | warning | info 2. A file path and line number 3. A one-sentence explanation 4. A concrete reproduction a test that would fail Output your findings as JSON to /workspace/review-output.json. Do not modify any source files. Do not run the test suite unless asked. Diff has ${DIFF SIZE} lines." Run Kiro in review mode kiro run \ --prompt "$PROMPT" \ --config /workspace/kiro.config.json \ --output /workspace/review-output.json Validate the output if jq -e '.findings' /workspace/review-output.json /dev/null 2 &1; then echo "ERROR: Kiro did not produce valid findings JSON" &2 exit 1 fi echo "Review complete: $ jq '.findings | length' /workspace/review-output.json findings" docker build -t kiro-sandbox:latest ./sandbox Why a separate Dockerfile? You could run Kiro directly on the runner, but then the agent has access to your CI secrets, your network, and your entire filesystem. The sandbox is the difference between "agentic review" and "agentic liability." Now we create the GitHub Actions workflow that triggers on every PR and runs the sandboxed agent. Create .github/workflows/agentic-review.yml : name: Agentic Code Review on: pull request: types: opened, synchronize, reopened permissions: contents: read pull-requests: write checks: write jobs: agentic-review: runs-on: ubuntu-latest timeout-minutes: 15 steps: - name: Checkout PR branch uses: actions/checkout@v4 with: fetch-depth: 0 - name: Set up Node uses: actions/setup-node@v4 with: node-version: '20' - name: Install dependencies run: npm ci - name: Build project early compile check run: npm run build continue-on-error: true - name: Generate PR diff run: | git diff origin/${{ github.base ref }}...HEAD diff.patch echo "Diff lines: $ wc -l < diff.patch " - name: Build sandbox image run: docker build -t kiro-sandbox:latest ./sandbox - name: Run sandboxed Kiro agent env: KIRO API KEY: ${{ secrets.KIRO API KEY }} run: | docker run --rm \ --name kiro-review \ --network none \ --memory 2g \ --cpus 2 \ --read-only \ --tmpfs /tmp \ -v "$PWD/diff.patch:/workspace/diff.patch:ro" \ -v "$PWD:/workspace/repo:ro" \ -v "$PWD/kiro.config.json:/workspace/kiro.config.json:ro" \ -v "$PWD/review-output.json:/workspace/review-output.json" \ -e KIRO API KEY="$KIRO API KEY" \ kiro-sandbox:latest continue-on-error: true - name: Run orchestrator run: node orchestrator.mjs - name: Post review comment if: always uses: actions/github-script@v7 with: script: | const fs = require 'fs' ; const findings = JSON.parse fs.readFileSync 'review-output.json', 'utf8' ; const summary = findings.findings || ; const critical = summary.filter f = f.severity === 'critical' ; const warning = summary.filter f = f.severity === 'warning' ; let body = 🤖 Agentic Review\n\n ; body += Findings: ${summary.length} ${critical.length} critical, ${warning.length} warning \n\n ; if summary.length === 0 { body += No issues found. ✅\n ; } else { body += | Severity | File | Line | Finding |\n|---|---|---|---|\n ; for const f of summary.slice 0, 20 { body += | ${f.severity} | \ ${f.file}\ | ${f.line} | ${f.message} |\n ; } } await github.rest.issues.createComment { owner: context.repo.owner, repo: context.repo.repo, issue number: context.issue.number, body: body } ; Key details: --network none — the agent cannot exfiltrate data or call external APIs. --read-only — the agent cannot modify the repo it only writes to the mounted review-output.json . --memory 2g --cpus 2 — caps resource usage so a runaway agent can't kill the runner. continue-on-error: true — if Kiro crashes, we still run the orchestrator so the PR gets an "inconclusive" comment instead of a silent failure. The orchestrator is the policy brain. It reads Kiro's JSON output, filters findings according to your rules, and emits a status check that GitHub branch protection can enforce. Create orchestrator.mjs : python import fs from 'node:fs'; import path from 'node:path'; const REVIEW OUTPUT = 'review-output.json'; // ---- Policy configuration ---- const POLICY = { // Severities that block the merge blocking: 'critical' , // Severities that are reported but do not block reportOnly: 'warning', 'info' , // Files to ignore e.g., generated code, lockfiles ignorePaths: /\.lock$/, /dist\//, /node modules\// , // Max findings to report before we consider the review "flooded" maxFindings: 50, // If Kiro crashed no valid output , treat as inconclusive, not failure inconclusiveOnError: true, }; function loadFindings { if fs.existsSync REVIEW OUTPUT { return { findings: , error: 'No review output produced' }; } try { const raw = fs.readFileSync REVIEW OUTPUT, 'utf8' ; const parsed = JSON.parse raw ; return { findings: parsed.findings || , error: null }; } catch e { return { findings: , error: Failed to parse: ${e.message} }; } } function applyPolicy findings { return findings .filter f = POLICY.ignorePaths.some re = re.test f.file || '' .slice 0, POLICY.maxFindings ; } function classify findings { const blockers = findings.filter f = POLICY.blocking.includes f.severity ; const reports = findings.filter f = POLICY.reportOnly.includes f.severity ; return { blockers, reports }; } function setStatusCheck conclusion, summary { // In GitHub Actions, write the status to a file that the workflow reads const payload = { conclusion, summary, blockers: summary.blockers, reports: summary.reports, }; fs.writeFileSync 'status-check.json', JSON.stringify payload, null, 2 ; console.log Status: ${conclusion} ; console.log Blockers: ${summary.blockers.length}, Reports: ${summary.reports.length} ; } // ---- Main ---- const { findings, error } = loadFindings ; if error && POLICY.inconclusiveOnError { setStatusCheck 'neutral', { blockers: , reports: , error, message: 'Agentic review was inconclusive — falling back to human review.', } ; process.exit 0 ; } const filtered = applyPolicy findings ; const { blockers, reports } = classify filtered ; if blockers.length 0 { setStatusCheck 'failure', { blockers, reports } ; process.exit 1 ; // This makes the GitHub Actions job fail → blocks merge } else { setStatusCheck 'success', { blockers, reports } ; process.exit 0 ; } blocking: 'critical' ignorePaths inconclusiveOnError neutral rather than failure . A broken agent should not block a good PR. The docker run command in the workflow is already fairly locked down, but for production you'll want to go further. Here's a hardened version: docker run --rm \ --name kiro-review \ --network none \ --memory 2g \ --memory-swap 2g \ --cpus 2 \ --pids-limit 256 \ --read-only \ --tmpfs /tmp:size=64m \ --tmpfs /run:size=16m \ --cap-drop ALL \ --security-opt no-new-privileges \ --security-opt seccomp=default \ -v "$PWD/diff.patch:/workspace/diff.patch:ro" \ -v "$PWD:/workspace/repo:ro" \ -v "$PWD/kiro.config.json:/workspace/kiro.config.json:ro" \ -v "$PWD/review-output.json:/workspace/review-output.json" \ -e KIRO API KEY="$KIRO API KEY" \ kiro-sandbox:latest | Flag | Purpose | |---|---| | --network none | No network egress — the agent can't call home or exfiltrate secrets | | --cap-drop ALL | Drops all Linux capabilities — no raw sockets, no ptrace, no mount | | --security-opt no-new-privileges | Prevents the agent from escalating privileges via setuid binaries | | --pids-limit 256 | Stops fork bombs | | --read-only | The root FS is read-only; the agent can only write to the mounted output file | | --tmpfs | Ephemeral writable space that vanishes on container exit | Important: Never mount the CI secrets directory into the sandbox. Pass only the API key via environment variable, and even then, consider using a short-lived, scoped token that Kiro can't reuse after the job ends. Now we make the review mandatory . GitHub branch protection rules are the enforcement mechanism. The orchestrator writes status-check.json , but GitHub Actions needs a real check. The cleanest way is to use the actions/github-script step to create a check run, or simply let the job's exit code drive a required status check. Add this to the workflow after the orchestrator step: js - name: Create status check if: always run: | node -e " const fs = require 'fs' ; const status = JSON.parse fs.readFileSync 'status-check.json', 'utf8' ; // The job itself already exits 1 on blockers, so GitHub marks it failed console.log 'Conclusion:', status.conclusion ; " Actually, the simplest robust approach: the orchestrator's process.exit 1 makes the GitHub Actions job fail. Then in branch protection, require the agentic-review check to pass. In GitHub → Settings → Branches → Add rule: Agentic Code Review check or the job name agentic-review . Now, if the agent finds a critical bug, the check fails, and the merge button is disabled. Kiro's "scope": "diff" is your friend. Reviewing the entire repo on every PR is slow and expensive. For a 500-line PR, the diff is the only thing that matters. Cap tokens and time: { "review": { "maxTokens": 8000, "timeoutMs": 120000, "maxFiles": 20, "maxDiffLines": 1000 } } If the diff exceeds the budget, the agent should review a sample and mark the review as "partial" — better than timing out. Building the Docker image on every PR is wasteful. Use docker/build-push-action with a cache, or push to a registry and pull it in CI: - name: Pull sandbox image run: docker pull ghcr.io/your-org/kiro-sandbox:latest Agentic review should augment human review, not replace it. Configure the orchestrator so that if a finding touches a file in CODEOWNERS with a critical path e.g., auth, payments , it always escalates to a human, even if the agent says it's fine. Log every finding to a metrics store. After a month, compute your false-positive rate findings that a human dismissed . If it's above 30%, tighten the prompt or raise severityThreshold . | Scenario | Desired behavior | Implementation | |---|---|---| | Kiro times out | Mark check as neutral , don't block | timeout-minutes + inconclusiveOnError: true | | Sandbox OOM | Retry once with lower memory, then neutral | --memory + retry logic in workflow | | Malicious PR tries to escape sandbox | Container is killed, job fails, no host compromise | --cap-drop ALL , --network none , --pids-limit | | Agent hallucinates a finding | Human dismisses it; we track it | False-positive metrics + allowlist | | Huge diff 1000 lines | Review a sample, mark as partial | maxDiffLines budget | | Kiro CLI not on runner | Fall back to human-only review | continue-on-error + neutral check | | Secrets in repo | Sandbox has no network, so exfiltration is impossible | --network none | The worst case is a PR that contains a Dockerfile or a script designed to break out of the container. The --cap-drop ALL , --security-opt no-new-privileges , and --network none flags make this very hard. But you should also ensure the host runner is itself ephemeral GitHub-hosted runners are so that even a successful escape only compromises a throwaway machine. No. It replaces the boring 80% — the mechanical "did you handle the empty array" checks. The human should focus on architecture, design intent, and API contracts. Configure branch protection to require both the agentic check and a human approval. Three strategies: critical finding. If the agent can't produce one, the finding is downgraded.