{"slug": "the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github", "title": "The Agentic Code Review Stack: Catching 'Almost Right' Bugs with Kiro CLI, GitHub Actions, and Sandboxed Agents", "summary": "A developer published a walkthrough of an agentic code review stack that combines Kiro CLI, AWS's agentic coding CLI, with GitHub Actions and Docker-sandboxed autonomous agents to catch \"almost right\" bugs that pass compilation, linting and unit tests. The pipeline runs the agent in an isolated container with no network and a read-only filesystem, lets it execute changed code and probe edge cases, and gates merges on the resulting findings.", "body_md": "*Originally published on [tamiz.pro](https://tamiz.pro/insights/agentic-code-review-stack-kiro-cli-github-actions-sandboxed-agents).*\n\nEvery engineer has shipped an \"almost right\" fix. The code compiles. The unit tests pass. The PR looks clean. Three weeks later, a production incident trace points straight to the one branch you didn't test — the null-coalescing chain that silently swallowed an empty array, or the race condition that only appears when two jobs run in parallel. These are not logic errors; they are *contextual* errors, and they are precisely what conventional review tooling misses.\n\nThis tutorial builds a complete agentic code review stack that catches \"almost right\" bugs before they reach `main`. You'll combine **Kiro CLI** (AWS's agentic coding CLI) for autonomous analysis, **GitHub Actions** for orchestration and gating, and **sandboxed autonomous agents** (Docker-isolated) for safe, repeatable execution. By the end, you'll have a merge-gated pipeline that reviews every PR with an agent that can actually *run* your code, explore the call graph, and flag the subtle semantic drift that linters and human eyes skip.\n\n**Why this stack?** Kiro CLI brings agentic reasoning to the terminal; GitHub Actions gives you a reliable trigger and a merge gate; the sandbox gives you the confidence to let an autonomous agent run untrusted code without burning down your infrastructure.\n\nLet's define the target precisely. An \"almost right\" bug is one that satisfies all *static* and *shallow* checks but violates a *runtime* or *contextual* invariant. It's not a typo, and it's not a missing semicolon — the compiler is happy, the linter is silent, and the existing test suite is green.\n\n| Flavor | Example | Why it slips through | \n|---|---|---|\n| **Semantic drift** | `if (items.length > 0)` instead of`if (items?.length)` | Type checker can't see the runtime `undefined` | \n| **Concurrency hazard** | Shared mutable state across async handlers | Unit tests run single-threaded | \n| **Contract violation** | Function returns `null` where the API contract says`[]` | No integration test exercises the boundary | \n\nHuman reviewers are bad at these because they review *diffs*, not *executions*. A human reads the changed lines; an agentic reviewer can execute the code, trace the data flow, and compare the behavior against the module's own contract.\n\nLinters (ESLint, Ruff, etc.) operate on the AST. They catch *patterns*, not *semantics*. TypeScript catches type mismatches, but it can't know that `Array.prototype.sort` mutates in place and that your caller relied on the original order. These tools are necessary but insufficient.\n\nAn autonomous agent with a sandbox can:\n\n`null`, `NaN`, huge numbers) to the changed functions.` main`.\nThat last capability — *writing a failing test* — is the killer feature. A bug that can be demonstrated by a failing test is a bug that cannot be argued away.\n\nBefore we write any code, make sure you have the following.\n\n`ubuntu-latest` runner with Docker).`pull-requests: write` and `checks: write` permissions (for posting comments and creating status checks).\n| Component | Role | \n|---|---|\n| **Kiro CLI** | The agentic \"brain\" — parses the diff, reasons about the code, produces structured findings | \n| **GitHub Actions** | The \"trigger and gate\" — fires on PR events, runs the pipeline, blocks merge on failure | \n| **Sandboxed agent** | The \"hands\" — wraps Kiro in an isolated Docker container with no network, read-only FS, and resource limits | \n| **Orchestrator** | The \"policy layer\" — parses Kiro's JSON output, filters by severity, applies rules, sets the status check | \n\nHere's the end-to-end flow, as a diagram-in-words:\n\n```\n[PR opened / updated]\n        │\n        ▼\n[GitHub Actions workflow: agentic-review]\n        │\n        ├── checkout PR branch\n        ├── build project (to catch compile errors early)\n        ├── pull sandbox image (cached)\n        ├── run sandboxed Kiro agent  ──►  [Docker: --network none, --read-only, --memory 2g]\n        │                                     │\n        │                                     └── writes review-output.json\n        ├── run orchestrator (policy layer)\n        │       ├── parse findings\n        │       ├── filter by severity / rule allowlist\n        │       └── emit status: success | failure | neutral\n        └── post PR comment + set status check\n```\n\nThe pipeline has four distinct layers:\n\nKeeping these layers separate is the key to maintainability. If you want to swap Kiro for another agent (e.g., `claude -p` or `aider`), you only touch the cognition layer. If you want to change the merge policy, you only touch the policy layer.\n\nKiro CLI is the headless, CI-friendly interface to the Kiro agentic environment. It lets you run an autonomous coding agent from the command line, which is exactly what we need for a PR review pipeline.\n\n```\n# Install globally via npm (recommended for CI)\nnpm install -g @aws/kiro-cli\n\n# Or via the official installer script\ncurl -fsSL https://kiro.dev/install.sh | bash\n\n# Verify\nkiro --version\n```\n\n**Note:** Kiro CLI is under active development. Pin a specific version in your CI (`npm install -g @aws/kiro-cli@<version>`) so your pipeline doesn't break when a new release ships.\n\nKiro needs credentials to call the underlying model. In CI, use a secret:\n\n```\nexport KIRO_API_KEY=\"${{ secrets.KIRO_API_KEY }}\"\nkiro auth login --api-key \"$KIRO_API_KEY\"\n```\n\nFor local development, you can use the interactive login:\n\n```\nkiro auth login\n```\n\nCreate a `kiro.config.json` at the repo root (or `.kiro/config.json`):\n\n```\n{\n  \"model\": \"kiro-1.5-pro\",\n  \"mode\": \"review\",\n  \"maxTokens\": 8000,\n  \"temperature\": 0.1,\n  \"timeoutMs\": 120000,\n  \"review\": {\n    \"scope\": \"diff\",\n    \"includeTests\": true,\n    \"severityThreshold\": \"warning\",\n    \"outputFormat\": \"json\"\n  }\n}\n```\n\nKey fields:\n\n`\"mode\": \"review\"` — tells Kiro to behave as a reviewer, not a code generator.`\"scope\": \"diff\"` — limits the agent's attention to the PR diff (critical for cost and latency).`\"outputFormat\": \"json\"` — gives us machine-readable findings for the orchestrator.\n\n```\n# Run Kiro against a single file to confirm it works\nkiro run --prompt \"Review src/utils/date.ts for edge-case bugs\" --output review.json\ncat review.json | jq '.findings | length'\n```\n\nIf this prints a number (even `0`), Kiro is working.\n\nNow we wrap Kiro in a Docker sandbox. The sandbox is what makes this stack *safe* — it means an autonomous agent can run arbitrary code from a PR without risking your CI infrastructure.\n\nCreate `sandbox/Dockerfile`:\n\n```\nFROM node:20-slim\n\n# Minimal tooling: git for diffing, jq for JSON parsing\nRUN apt-get update \\\n    && apt-get install -y --no-install-recommends git jq \\\n    && rm -rf /var/lib/apt/lists/*\n\n# Install Kiro CLI inside the sandbox\nRUN npm install -g @aws/kiro-cli@1.2.3\n\n# The agent will run as a non-root user\nRUN useradd -m -u 1000 agent\nUSER agent\n\nWORKDIR /workspace\n\n# Default entrypoint: the agent runner script\nCOPY --chown=agent:agent run-agent.sh /usr/local/bin/run-agent.sh\nRUN chmod +x /usr/local/bin/run-agent.sh\n\nENTRYPOINT [\"run-agent.sh\"]\n```\n\nCreate `sandbox/run-agent.sh`:\n\n``` bash\n#!/usr/bin/env bash\nset -euo pipefail\n\n# The PR diff is mounted at /workspace/diff.patch\n# The repo is mounted read-only at /workspace/repo\n\nexport KIRO_API_KEY=\"${KIRO_API_KEY:?KIRO_API_KEY is required}\"\n\ncd /workspace/repo\n\n# Generate the prompt from the diff\nDIFF_SIZE=$(wc -l < /workspace/diff.patch)\nPROMPT=\"You are a senior code reviewer. Review the changes in /workspace/diff.patch.\nFocus ONLY on bugs that would pass existing tests but fail in production:\n- semantic drift (e.g., null vs empty array)\n- concurrency hazards (shared mutable state)\n- contract violations (return type vs API contract)\n- edge cases (empty inputs, boundary values)\n\nFor each finding, provide:\n1. A severity (critical | warning | info)\n2. A file path and line number\n3. A one-sentence explanation\n4. A concrete reproduction (a test that would fail)\n\nOutput your findings as JSON to /workspace/review-output.json.\nDo not modify any source files. Do not run the test suite unless asked.\nDiff has ${DIFF_SIZE} lines.\"\n\n# Run Kiro in review mode\nkiro run \\\n  --prompt \"$PROMPT\" \\\n  --config /workspace/kiro.config.json \\\n  --output /workspace/review-output.json\n\n# Validate the output\nif ! jq -e '.findings' /workspace/review-output.json > /dev/null 2>&1; then\n  echo \"ERROR: Kiro did not produce valid findings JSON\" >&2\n  exit 1\nfi\n\necho \"Review complete: $(jq '.findings | length' /workspace/review-output.json) findings\"\ndocker build -t kiro-sandbox:latest ./sandbox\n```\n\n**Why a separate Dockerfile?** You *could* run Kiro directly on the runner, but then the agent has access to your CI secrets, your network, and your entire filesystem. The sandbox is the difference between \"agentic review\" and \"agentic liability.\"\n\nNow we create the GitHub Actions workflow that triggers on every PR and runs the sandboxed agent.\n\nCreate `.github/workflows/agentic-review.yml`:\n\n```\nname: Agentic Code Review\n\non:\n  pull_request:\n    types: [opened, synchronize, reopened]\n\npermissions:\n  contents: read\n  pull-requests: write\n  checks: write\n\njobs:\n  agentic-review:\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    steps:\n      - name: Checkout PR branch\n        uses: actions/checkout@v4\n        with:\n          fetch-depth: 0\n\n      - name: Set up Node\n        uses: actions/setup-node@v4\n        with:\n          node-version: '20'\n\n      - name: Install dependencies\n        run: npm ci\n\n      - name: Build project (early compile check)\n        run: npm run build\n        continue-on-error: true\n\n      - name: Generate PR diff\n        run: |\n          git diff origin/${{ github.base_ref }}...HEAD > diff.patch\n          echo \"Diff lines: $(wc -l < diff.patch)\"\n\n      - name: Build sandbox image\n        run: docker build -t kiro-sandbox:latest ./sandbox\n\n      - name: Run sandboxed Kiro agent\n        env:\n          KIRO_API_KEY: ${{ secrets.KIRO_API_KEY }}\n        run: |\n          docker run --rm \\\n            --name kiro-review \\\n            --network none \\\n            --memory 2g \\\n            --cpus 2 \\\n            --read-only \\\n            --tmpfs /tmp \\\n            -v \"$PWD/diff.patch:/workspace/diff.patch:ro\" \\\n            -v \"$PWD:/workspace/repo:ro\" \\\n            -v \"$PWD/kiro.config.json:/workspace/kiro.config.json:ro\" \\\n            -v \"$PWD/review-output.json:/workspace/review-output.json\" \\\n            -e KIRO_API_KEY=\"$KIRO_API_KEY\" \\\n            kiro-sandbox:latest\n        continue-on-error: true\n\n      - name: Run orchestrator\n        run: node orchestrator.mjs\n\n      - name: Post review comment\n        if: always()\n        uses: actions/github-script@v7\n        with:\n          script: |\n            const fs = require('fs');\n            const findings = JSON.parse(fs.readFileSync('review-output.json', 'utf8'));\n            const summary = findings.findings || [];\n            const critical = summary.filter(f => f.severity === 'critical');\n            const warning = summary.filter(f => f.severity === 'warning');\n\n            let body = `## 🤖 Agentic Review\\n\\n`;\n            body += `**Findings:** ${summary.length} (${critical.length} critical, ${warning.length} warning)\\n\\n`;\n            if (summary.length === 0) {\n              body += `No issues found. ✅\\n`;\n            } else {\n              body += `| Severity | File | Line | Finding |\\n|---|---|---|---|\\n`;\n              for (const f of summary.slice(0, 20)) {\n                body += `| ${f.severity} | \\`${f.file}\\` | ${f.line} | ${f.message} |\\n`;\n              }\n            }\n\n            await github.rest.issues.createComment({\n              owner: context.repo.owner,\n              repo: context.repo.repo,\n              issue_number: context.issue.number,\n              body: body\n            });\n```\n\n**Key details:**\n\n`--network none` — the agent cannot exfiltrate data or call external APIs.`--read-only` — the agent cannot modify the repo (it only writes to the mounted `review-output.json`).`--memory 2g --cpus 2` — caps resource usage so a runaway agent can't kill the runner.`continue-on-error: true` — if Kiro crashes, we still run the orchestrator so the PR gets an \"inconclusive\" comment instead of a silent failure.\nThe orchestrator is the policy brain. It reads Kiro's JSON output, filters findings according to your rules, and emits a status check that GitHub branch protection can enforce.\n\nCreate `orchestrator.mjs`:\n\n``` python\nimport fs from 'node:fs';\nimport path from 'node:path';\n\nconst REVIEW_OUTPUT = 'review-output.json';\n\n// ---- Policy configuration ----\nconst POLICY = {\n  // Severities that block the merge\n  blocking: ['critical'],\n  // Severities that are reported but do not block\n  reportOnly: ['warning', 'info'],\n  // Files to ignore (e.g., generated code, lockfiles)\n  ignorePaths: [/\\.lock$/, /dist\\//, /node_modules\\//],\n  // Max findings to report before we consider the review \"flooded\"\n  maxFindings: 50,\n  // If Kiro crashed (no valid output), treat as inconclusive, not failure\n  inconclusiveOnError: true,\n};\n\nfunction loadFindings() {\n  if (!fs.existsSync(REVIEW_OUTPUT)) {\n    return { findings: [], error: 'No review output produced' };\n  }\n  try {\n    const raw = fs.readFileSync(REVIEW_OUTPUT, 'utf8');\n    const parsed = JSON.parse(raw);\n    return { findings: parsed.findings || [], error: null };\n  } catch (e) {\n    return { findings: [], error: `Failed to parse: ${e.message}` };\n  }\n}\n\nfunction applyPolicy(findings) {\n  return findings\n    .filter((f) => !POLICY.ignorePaths.some((re) => re.test(f.file || '')))\n    .slice(0, POLICY.maxFindings);\n}\n\nfunction classify(findings) {\n  const blockers = findings.filter((f) => POLICY.blocking.includes(f.severity));\n  const reports = findings.filter((f) => POLICY.reportOnly.includes(f.severity));\n  return { blockers, reports };\n}\n\nfunction setStatusCheck(conclusion, summary) {\n  // In GitHub Actions, write the status to a file that the workflow reads\n  const payload = {\n    conclusion,\n    summary,\n    blockers: summary.blockers,\n    reports: summary.reports,\n  };\n  fs.writeFileSync('status-check.json', JSON.stringify(payload, null, 2));\n  console.log(`Status: ${conclusion}`);\n  console.log(`Blockers: ${summary.blockers.length}, Reports: ${summary.reports.length}`);\n}\n\n// ---- Main ----\nconst { findings, error } = loadFindings();\n\nif (error && POLICY.inconclusiveOnError) {\n  setStatusCheck('neutral', {\n    blockers: [],\n    reports: [],\n    error,\n    message: 'Agentic review was inconclusive — falling back to human review.',\n  });\n  process.exit(0);\n}\n\nconst filtered = applyPolicy(findings);\nconst { blockers, reports } = classify(filtered);\n\nif (blockers.length > 0) {\n  setStatusCheck('failure', { blockers, reports });\n  process.exit(1); // This makes the GitHub Actions job fail → blocks merge\n} else {\n  setStatusCheck('success', { blockers, reports });\n  process.exit(0);\n}\n```\n\n`blocking: ['critical']`` ignorePaths``inconclusiveOnError`` neutral` rather than `failure`. A broken agent should not block a good PR.\nThe `docker run` command in the workflow is already fairly locked down, but for production you'll want to go further. Here's a hardened version:\n\n```\ndocker run --rm \\\n  --name kiro-review \\\n  --network none \\\n  --memory 2g \\\n  --memory-swap 2g \\\n  --cpus 2 \\\n  --pids-limit 256 \\\n  --read-only \\\n  --tmpfs /tmp:size=64m \\\n  --tmpfs /run:size=16m \\\n  --cap-drop ALL \\\n  --security-opt no-new-privileges \\\n  --security-opt seccomp=default \\\n  -v \"$PWD/diff.patch:/workspace/diff.patch:ro\" \\\n  -v \"$PWD:/workspace/repo:ro\" \\\n  -v \"$PWD/kiro.config.json:/workspace/kiro.config.json:ro\" \\\n  -v \"$PWD/review-output.json:/workspace/review-output.json\" \\\n  -e KIRO_API_KEY=\"$KIRO_API_KEY\" \\\n  kiro-sandbox:latest\n```\n\n| Flag | Purpose | \n|---|---|\n| `--network none` | No network egress — the agent can't call home or exfiltrate secrets | \n| `--cap-drop ALL` | Drops all Linux capabilities — no raw sockets, no ptrace, no mount | \n| `--security-opt no-new-privileges` | Prevents the agent from escalating privileges via setuid binaries | \n| `--pids-limit 256` | Stops fork bombs | \n| `--read-only` | The root FS is read-only; the agent can only write to the mounted output file | \n| `--tmpfs` | Ephemeral writable space that vanishes on container exit | \n\n**Important:** Never mount the CI secrets directory into the sandbox. Pass only the API key via environment variable, and even then, consider using a short-lived, scoped token that Kiro can't reuse after the job ends.\n\nNow we make the review *mandatory*. GitHub branch protection rules are the enforcement mechanism.\n\nThe orchestrator writes `status-check.json`, but GitHub Actions needs a real check. The cleanest way is to use the `actions/github-script` step to create a check run, or simply let the job's exit code drive a required status check.\n\nAdd this to the workflow after the orchestrator step:\n\n``` js\n- name: Create status check\n  if: always()\n  run: |\n    node -e \"\n      const fs = require('fs');\n      const status = JSON.parse(fs.readFileSync('status-check.json', 'utf8'));\n      // The job itself already exits 1 on blockers, so GitHub marks it failed\n      console.log('Conclusion:', status.conclusion);\n    \"\n```\n\nActually, the simplest robust approach: the orchestrator's `process.exit(1)` makes the GitHub Actions job fail. Then in branch protection, require the `agentic-review` check to pass.\n\nIn GitHub → Settings → Branches → Add rule:\n\n`Agentic Code Review` check (or the job name `agentic-review`).\nNow, if the agent finds a critical bug, the check fails, and the merge button is disabled.\n\nKiro's `\"scope\": \"diff\"` is your friend. Reviewing the entire repo on every PR is slow and expensive. For a 500-line PR, the diff is the only thing that matters.\n\nCap tokens and time:\n\n```\n{\n  \"review\": {\n    \"maxTokens\": 8000,\n    \"timeoutMs\": 120000,\n    \"maxFiles\": 20,\n    \"maxDiffLines\": 1000\n  }\n}\n```\n\nIf the diff exceeds the budget, the agent should review a *sample* and mark the review as \"partial\" — better than timing out.\n\nBuilding the Docker image on every PR is wasteful. Use `docker/build-push-action` with a cache, or push to a registry and pull it in CI:\n\n```\n- name: Pull sandbox image\n  run: docker pull ghcr.io/your-org/kiro-sandbox:latest\n```\n\nAgentic review should *augment* human review, not replace it. Configure the orchestrator so that if a finding touches a file in `CODEOWNERS` with a `critical` path (e.g., auth, payments), it always escalates to a human, even if the agent says it's fine.\n\nLog every finding to a metrics store. After a month, compute your false-positive rate (findings that a human dismissed). If it's above 30%, tighten the prompt or raise `severityThreshold`.\n\n| Scenario | Desired behavior | Implementation | \n|---|---|---|\n| Kiro times out | Mark check as `neutral` , don't block | `timeout-minutes` +`inconclusiveOnError: true` | \n| Sandbox OOM | Retry once with lower memory, then `neutral` | `--memory` + retry logic in workflow | \n| Malicious PR tries to escape sandbox | Container is killed, job fails, no host compromise | `--cap-drop ALL` ,`--network none` ,`--pids-limit` | \n| Agent hallucinates a finding | Human dismisses it; we track it | False-positive metrics + allowlist | \n| Huge diff (>1000 lines) | Review a sample, mark as partial | `maxDiffLines` budget | \n| Kiro CLI not on runner | Fall back to human-only review | `continue-on-error` +`neutral` check | \n| Secrets in repo | Sandbox has no network, so exfiltration is impossible | `--network none` | \n\nThe worst case is a PR that contains a Dockerfile or a script designed to break out of the container. The `--cap-drop ALL`, `--security-opt no-new-privileges`, and `--network none` flags make this very hard. But you should also ensure the *host* runner is itself ephemeral (GitHub-hosted runners are) so that even a successful escape only compromises a throwaway machine.\n\nNo. It *replaces the boring 80%* — the mechanical \"did you handle the empty array\" checks. The human should focus on architecture, design intent, and API contracts. Configure branch protection to require *both* the agentic check and a human approval.\n\nThree strategies:\n\n`critical` finding. If the agent can't produce one, the finding is downgraded.", "url": "https://wpnews.pro/news/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github", "canonical_source": "https://dev.to/tamizuddin/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github-actions-and-957", "published_at": "2026-11-12 18:13:23+00:00", "updated_at": "2026-11-12 18:23:03.136933+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "mlops", "artificial-intelligence"], "entities": ["Kiro CLI", "AWS", "GitHub Actions", "Docker", "ESLint", "Ruff", "TypeScript"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github", "markdown": "https://wpnews.pro/news/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github.md", "text": "https://wpnews.pro/news/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github.txt", "jsonld": "https://wpnews.pro/news/the-agentic-code-review-stack-catching-almost-right-bugs-with-kiro-cli-github.jsonld"}}