{"slug": "show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments", "title": "Show HN: AI PR reviewer built to stay quiet – 89% of merged PRs got no comments", "summary": "Pr-sage, an open-source AI pull request reviewer, reports that 89% of merged PRs received no comments, achieving its goal of eliminating review noise. The tool, available as a CLI, GitHub Action, and TypeScript library, uses content fingerprints to prevent duplicate comments, reviews only new commits incrementally, and supports providers including Claude, OpenAI, and Gemini, with options for self-hosted endpoints. It also offers a quality gate via --fail-on critical and can review local diffs before pushing.", "body_md": "**An AI PR reviewer built to eliminate review noise — not add to it.**\n\nMost AI reviewers re-review the whole PR on every push and repeat themselves until the team mutes them. pr-sage is designed around the opposite goal: say each thing once, follow your team's rules, and stay silent when there is nothing new to say.\n\n- 🔇\n**Zero duplicate comments.** Findings carry content fingerprints — a line shift won't make the same comment appear twice, and re-runs post nothing when nothing changed. - ✅\n**Finding lifecycle.** Follow-up reviews report which findings remain unresolved and which were fixed. - ⏩\n**Incremental by default.** After the first review, only the commits you pushed since get reviewed. Less noise, fewer tokens. - 📏\n**Your rules, not generic advice.**`.pr-sage.json`\n\ninstructions plus automatic`CLAUDE.md`\n\n/`CONTRIBUTING.md`\n\ninjection make reviews follow team conventions. - 🚦\n**A quality gate, not just commentary.**`--fail-on critical`\n\nblocks merges;`--event auto`\n\napproves clean PRs and requests changes on real problems. - 🖥️\n**Reviews before the PR exists.**`pr-sage local`\n\nreviews your`git diff`\n\npre-push — no server, no PR, no GitHub token. - 🔐\n**Your keys, your data path.** No server, nothing stored; code goes only to the provider you choose —**Claude, OpenAI, or Gemini**— or never leaves your machine at all with a** self-hosted OpenAI-compatible endpoint**(Ollama, vLLM, LM Studio). See[SECURITY.md](/Kyeom1997/pr-sage/blob/main/SECURITY.md).\n\nShips as a **CLI**, a **GitHub Action**, and a **TypeScript library**.\n\nThe fastest path — an interactive wizard that writes your config and the GitHub Action workflow, and tells you exactly which secret to register:\n\n```\nnpx pr-sage init\nnpx pr-sage doctor\n```\n\nOr by hand (CLI):\n\n```\nexport GITHUB_TOKEN=ghp_...\nexport ANTHROPIC_API_KEY=sk-ant-...\n\nnpx pr-sage review --repo owner/name --pr 123\n```\n\nPreview without posting anything:\n\n```\nnpx pr-sage review --repo owner/name --pr 123 --dry-run\n```\n\nReview your local changes before pushing (no PR, no GitHub token needed):\n\n```\nnpx pr-sage local --base main            # diff vs main\nnpx pr-sage local --staged --fail-on critical   # gate staged changes\n```\n\nReview in Korean with a different provider:\n\n```\nexport OPENAI_API_KEY=sk-...\nnpx pr-sage review --repo owner/name --pr 123 --provider openai --locale Korean\n```\n\n| Option | Default | Description |\n|---|---|---|\n`-p, --pr <number>` |\n(required) | Pull request number |\n`-r, --repo <owner/name>` |\n`$GITHUB_REPOSITORY` |\nTarget repository |\n`--provider <name>` |\n`anthropic` |\n`anthropic` | `openai` | `gemini` |\n`-m, --model <id>` |\nprovider default | Model id (`claude-opus-4-8` , `gpt-5` , `gemini-flash-latest` ) |\n`--locale <lang>` |\n`English` |\nLanguage for the review output; `auto` detects it from the PR title/body |\n`--paths <globs>` |\n— | Only review files matching these comma-separated globs (monorepo scoping) |\n`--max-tokens <n>` |\n— | Cost guard: stop launching new batches once this many tokens are spent |\n`--force` |\n— | Review even draft, WIP-titled, or `skip-review` -labeled PRs (skipped by default) |\n`--exclude <patterns>` |\n— | Comma-separated globs or substrings to skip (added to defaults: lockfiles, `dist/` , `build/` , …) |\n`--min-severity <sev>` |\n— | Drop findings below this severity (e.g. `suggestion` hides nitpicks) |\n`--fail-on <sev>` |\n— | Exit 1 if any finding is at or above this severity — use as a CI quality gate |\n`--context <mode>` |\n`patch` |\n`full` sends complete file contents to the model for better accuracy (more tokens) |\n`--event <mode>` |\n`comment` |\n`auto` approves clean PRs and requests changes on critical findings (falls back to comment on your own PRs) |\n`--verify` |\noff | Second model pass that rejects unconfirmed findings |\n`--verify-provider <name>` |\nsame provider | Use a separate provider for verification |\n`--verify-model <id>` |\nprovider default | Use a separate verification model |\n`--verify-failure <mode>` |\n`abort` |\n`abort` , `keep` , or `drop` when verification fails |\n`--output <format>` |\n`text` |\n`json` or `sarif` for machine-readable results |\n`--fail-on-incomplete` |\noff | Fail when filtering, missing patches, or the token budget leaves part of the change unreviewed |\n`--check-run` |\noff | Publish findings as GitHub Check Run annotations |\n`--no-dedupe` |\n— | Repost findings already commented by a previous pr-sage review (dedup is on by default) |\n`--no-incremental` |\n— | Always review the full PR diff instead of only commits since the last pr-sage review |\n`--batch-chars <n>` |\n`80000` |\nMax diff characters per model request; larger PRs are reviewed in batches |\n`--config <path>` |\n`.pr-sage.json` |\nConfig file path |\n`--dry-run` |\n— | Print the review to stdout instead of posting |\n\nRequired environment variables: `GITHUB_TOKEN`\n\n(with `pull_requests: write`\n\n), plus the API key for your provider (`ANTHROPIC_API_KEY`\n\n, `OPENAI_API_KEY`\n\n, or `GEMINI_API_KEY`\n\n).\n\nOn repeat runs (e.g. new commits pushed to the PR), pr-sage reviews **only the commits pushed since its last review** (incremental mode), skips findings it has already commented, and posts nothing when there is nothing new — no duplicate-comment spam, no wasted tokens. If your repo has a `CLAUDE.md`\n\nor `CONTRIBUTING.md`\n\n, it is automatically injected as review context (disable with `\"repoContext\": false`\n\n). GitHub Enterprise works out of the box via `$GITHUB_API_URL`\n\nor the `githubApiUrl`\n\nconfig field.\n\nEach run prints its token usage to stderr (`LLM usage: N call(s), X input / Y output tokens`\n\n) so cost stays visible.\nEvery summary also reports review coverage. Partial reviews never auto-approve a PR. Use\n`--fail-on-incomplete`\n\nwhen incomplete coverage must fail the CI quality gate.\n\nPoint the OpenAI provider at any OpenAI-compatible server and private code never leaves your machine — no API key required:\n\n```\nollama pull qwen2.5-coder:14b\nOPENAI_BASE_URL=http://localhost:11434/v1 \\\n  npx pr-sage review --repo owner/repo --pr 123 --provider openai --model qwen2.5-coder:14b\n```\n\nWorks the same with vLLM, LM Studio, or any gateway that speaks the OpenAI chat completions API.\nFor GitHub Actions, `init --provider self-hosted`\n\ngenerates a workflow for a\n`self-hosted`\n\nrunner; `localhost`\n\nmust refer to that runner, not a GitHub-hosted VM.\n\nPut a `.pr-sage.json`\n\nin the directory you run from (CLI flags override it):\n\n```\n{\n  \"provider\": \"anthropic\",\n  \"locale\": \"auto\",\n  \"exclude\": [\"src/generated/**\", \"**/*.snap\"],\n  \"paths\": [\"packages/web/**\"],\n  \"minSeverity\": \"suggestion\",\n  \"failOn\": \"critical\",\n  \"context\": \"full\",\n  \"maxTokensPerRun\": 200000,\n  \"failOnIncomplete\": true,\n  \"skipLabels\": [\"skip-review\"],\n  \"verify\": true,\n  \"verifyProvider\": \"gemini\",\n  \"verifyModel\": \"gemini-flash-latest\",\n  \"verifyFailure\": \"abort\",\n  \"checkRun\": true,\n  \"pathRules\": [\n    {\n      \"paths\": [\"packages/api/**\"],\n      \"instructions\": \"Check public API backward compatibility.\",\n      \"minSeverity\": \"suggestion\",\n      \"failOn\": \"warning\"\n    }\n  ],\n  \"instructions\": \"We use Result<T, E> for error handling — flag thrown exceptions in domain code. Prefer early returns over nested conditionals.\"\n}\n```\n\n`instructions`\n\nis injected into the review prompt — use it for team conventions the reviewer should enforce.\n\n```\n# .github/workflows/pr-sage.yml\nname: AI Review\non:\n  pull_request:\n    types: [opened, synchronize]\n\npermissions:\n  contents: read\n  pull-requests: write\n  checks: write\n\nconcurrency:\n  group: pr-sage-${{ github.event.pull_request.number }}\n  cancel-in-progress: true\n\njobs:\n  review:\n    runs-on: ubuntu-latest\n    steps:\n      # Read .pr-sage.json from trusted base code, never PR-controlled code.\n      - uses: actions/checkout@v4\n        with:\n          ref: ${{ github.event.pull_request.base.sha }}\n          persist-credentials: false\n      - uses: Kyeom1997/pr-sage@v1\n        with:\n          provider: anthropic\n          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}\n          locale: Korean\n          fail-on: critical   # optional: block merge on critical findings\n          fail-on-incomplete: \"true\"\n          check-run: \"true\"\n```\n\nTo upload SARIF, add `security-events: write`\n\nand set `sarif: \"true\"`\n\n. The Action\nalso exposes first-class `paths`\n\n, `max-tokens`\n\n, `verify-provider`\n\n,\n`verify-model`\n\n, `verify-failure`\n\n, and `openai-base-url`\n\ninputs.\n\nThe base-SHA checkout is deliberate: configuration and repository guidelines must come from trusted base code. The PR diff itself is always fetched through the GitHub API, and pr-sage rechecks the head SHA immediately before posting so a slow review cannot comment on a superseded commit.\n\n`scripts/bench.mjs`\n\nruns pr-sage (dry, nothing posted) over recent merged PRs of any public repo and records findings, severity mix, latency, and token usage, plus a labeling sheet for computing the valid-review rate:\n\n```\nnode scripts/bench.mjs --repos fastify/fastify --per-repo 5 --provider gemini\n```\n\n`--mode recall`\n\nmeasures the other axis: it picks merged PRs that received human review comments, reviews each PR's **first commit** (the state humans reviewed), and reports how many human-flagged locations pr-sage also flags — with a side-by-side sheet for manual verification.\n\nThe `Quality Benchmark`\n\nworkflow can run either benchmark manually and uploads\nthe generated JSON and labeling sheet as workflow artifacts.\n\n28 recently merged PRs across **fastify, hono, GitHub CLI, and Vite** (`gemini-flash-lite`\n\n, zero run failures):\n\n**25/28 (89%) produced zero comments**— quiet on code that had already passed human review. That silence is the point: no noise on clean diffs.- The other 3 PRs got 7 findings. Verifying each claim against the actual diff:\n**3/7 valid overall, 2/3 for**— the noise concentrated in the`warning`\n\nseverity`suggestion`\n\ntier. - Re-running those PRs with\nkept exactly the 2 diff-confirmed-valid findings (a real GPG-signing regression question in a deployment workflow) and rejected every invalid one.`--verify`\n\n**Median 2.3 s and ~3.5 k input tokens per PR**(≈ $0.01 for all 28 PRs at flash-lite list pricing).\n\nRaw results and the per-finding verification notes live in [ bench-results/](/Kyeom1997/pr-sage/blob/main/bench-results). Caveats: small sample, and merged-PR sampling measures\n\n*noise*, not\n\n*recall*— a detection benchmark (reviewing pre-review commits of PRs that later got human fixes) is future work.\n\n- Fetches the PR metadata and per-file patches from the GitHub API.\n- Annotates both sides of the diff — added/context lines with new-file numbers, deleted lines with old-file numbers — so findings can anchor to removed code too (e.g. \"this deleted validation was still needed\"). Lockfiles/build artifacts are filtered out.\n- Asks the LLM for a structured review (JSON schema — no parsing heuristics): summary + findings with\n`path`\n\n,`line`\n\n,`severity`\n\n, and an optional single-line suggestion. - Validates every finding at runtime (zod) and against the diff (GitHub rejects reviews that comment on lines outside the diff), demotes unsafe multi-line suggestions, skips findings already posted by a previous run, retries on provider rate limits, and posts one review: inline comments + summary.\n\nSeverities: 🔴 critical · 🟡 warning · 🔵 suggestion · ⚪ nitpick. Safe single-line fixes are posted as GitHub suggestion blocks you can commit with one click.\n\n``` js\nimport { GitHubClient, createProvider, runReview } from \"pr-sage\";\n\nconst github = new GitHubClient(process.env.GITHUB_TOKEN!, \"owner\", \"repo\");\nconst pr = await github.fetchPullRequest(123);\nconst provider = createProvider(\"anthropic\");\nconst { result } = await runReview(provider, pr, {\n  locale: \"English\",\n  exclude: [],\n  batchCharBudget: 80_000,\n  log: console.error,\n});\n```\n\nReviewing code means sending diffs (and optionally full files) to the LLM provider you choose — read [SECURITY.md](/Kyeom1997/pr-sage/blob/main/SECURITY.md) for the exact data flow, provider policy links, prompt-injection mitigations, and token scope guidance before enabling this on private repositories. The GitHub Action executes the bundled code committed at the tag you pin (no install step), and npm releases carry provenance.\n\nMIT", "url": "https://wpnews.pro/news/show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments", "canonical_source": "https://github.com/Kyeom1997/pr-sage", "published_at": "2026-08-19 08:20:28+00:00", "updated_at": "2026-08-19 08:43:00.301667+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "artificial-intelligence"], "entities": ["pr-sage", "GitHub", "Claude", "OpenAI", "Gemini", "Ollama", "vLLM", "LM Studio"], "alternates": {"html": "https://wpnews.pro/news/show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments", "markdown": "https://wpnews.pro/news/show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments.md", "text": "https://wpnews.pro/news/show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments.txt", "jsonld": "https://wpnews.pro/news/show-hn-ai-pr-reviewer-built-to-stay-quiet-89-of-merged-prs-got-no-comments.jsonld"}}