{"slug": "sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land", "title": "Sentinel – opens a PR to fix Stripe breaking changes before they land", "summary": "Sentinel, a new open-source tool, detects breaking changes in vendor API changelogs before they affect customer codebases, scans repositories for affected usages, and opens a pull request with fix diffs before the sunset date. The tool currently supports Stripe and TypeScript/JavaScript repos, with phases 1–6 built, and requires only Node to detect a real breaking change without an account or API key.", "body_md": "Watches vendor API changelogs, detects breaking changes before they hit customer codebases, scans a customer repo for affected usages, generates reviewable fix diffs, and opens a PR — before the sunset date, not after something breaks in production.\n\nSee it detect a real breaking change with no account, API key, or database — just Node:\n\n```\ngit clone https://github.com/rtsdque/sentinel.git\ncd sentinel\nnpm install\nnpx tsx src/cli.ts stripe --track acacia\n```\n\nThat's it — this scrapes Stripe's real changelog and prints a fully structured breaking-change\nevent to your terminal, deterministically (no LLM call). Everything past this point (interpreting\nchanges with an LLM, scanning a codebase, opening PRs, running on a schedule) layers on top and\nneeds progressively more setup — jump to the phase you want below, or `npm test`\n\nto run the\nsuite (also needs nothing but Node).\n\nThis is a real diff Sentinel generated and opened as a PR, unedited, against a test repo — Stripe\n[nested three Balance Settings request fields under a new required payments object](https://docs.stripe.com/changelog/basil/2025-08-27/balance-settings-nested-in-payments-field.md),\nand Sentinel found the affected call site and proposed the migration on its own:\n\n```\n export async function enableNegativeBalanceDebits() {\n   return stripe.balanceSettings.update({\n-    debit_negative_balances: true,\n-    payouts: {\n-      schedule: { interval: \"daily\" },\n+    payments: {\n+      debit_negative_balances: true,\n+      payouts: {\n+        schedule: { interval: \"daily\" },\n+      },\n     },\n   });\n }\n```\n\nThe PR body links the upstream changelog entry, states the deadline if there is one, and separately\nlists anything it *couldn't* confidently fix — see [Phase 4](#phase-4--fix-generator) and\n[Phase 5](#phase-5--pr-bot) below for what makes it refuse rather than guess.\n\nReasonable question — here's what's actually different, based on a real look at the landscape, not a guess:\n\n| Tool | What it does | Where it stops |\n|---|---|---|\nDependabot (incl. AI-agent remediation) |\nReacts to a known CVE or a new package version in your manifest | Anchored to your dependency file — never reads a vendor's changelog prose, only version diffs |\nRenovate |\nSame, plus `customDatasources` for arbitrary version feeds |\nStill just version tracking — no interpretation of what changed or why it breaks you |\noasdiff / openapi-diff |\nDiffs two OpenAPI specs, flags breaking changes precisely | Needs the vendor's own spec-versioned CI — doesn't touch your codebase at all |\nVisualping / Theneo / PageCrawl |\nWatch a changelog page, alert on any change | Stops at a notification. You still read it, find the usages, and fix them yourself |\nChangeSpec |\nAn open JSON format for vendors to publish structured change events | A spec waiting for vendor adoption — Stripe doesn't publish in it today, so someone still has to read the prose |\n\nEvery one of these is genuinely good at one piece. None of them do the thing this YC RFS actually\nasked for: watch a vendor's *unstructured* changelog prose, turn it into a structured breaking\nchange, find every place *your specific codebase* uses the affected surface, and propose the fix —\nbefore your build breaks, not after. That's the gap Sentinel targets, and as far as we could find\nduring a real competitive pass (not a hunch), nobody had shipped the full pipeline yet.\n\n**Phases 1–6 built**, scoped to Stripe + TypeScript/JavaScript customer repos only (see project\nscope notes — deliberately not broad).\n\n| Phase | What it does | Entry point |\n|---|---|---|\n| 1. Change Watcher | Scrapes Stripe's changelog, deterministically parses breaking-change entries into structured `ChangeEvent` s |\n`src/cli.ts` |\n| 2. Change Interpreter | LLM step (`claude-opus-4-8` ) that decomposes prose + the parsed param table into atomic, machine-actionable changes |\n`src/cli.ts --interpret` |\n| 3. Codebase Scanner | AST-based (`ts-morph` ) scan of a TS/JS repo for usages of the affected API surface, scoped to real Stripe client calls |\n`src/scan.ts` |\n| 4. Fix Generator | Per-match LLM call that proposes a diff or explicitly declines with a reason — never guesses | `src/fix.ts` |\n| 5. PR Bot | GitHub App that applies approved fixes, formats with Prettier, and opens a PR — every PR states what wasn't auto-fixed | `src/open-pr.ts` |\n| 6. Persistence & Scheduling | Postgres-backed \"have I seen this\" store + a repo registry, so a run only acts on genuinely new changes | `src/run-scheduled.ts` |\n\nNothing in this pipeline auto-merges. Every fix is generated for human/PR review; the PR body explicitly separates \"proposed changes\" from \"needs manual review,\" because there's no customer test suite to verify correctness against.\n\n```\nsrc/\n  lib/                 fetch wrapper, markdown-table parsing\n  watcher/stripe/       Phase 1: changelog fetch + parse + ChangeEvent builder\n  interpreter/stripe.ts Phase 2: LLM decomposition into atomic changes\n  scanner/typescript.ts Phase 3: ts-morph AST scan for affected usages\n  fixer/stripe.ts       Phase 4: per-match fix generation + apply/consolidate/format\n  github/\n    app.ts               GitHub App installation auth\n    pr.ts                 branch + commit + PR creation\n  cli.ts / scan.ts / fix.ts / open-pr.ts   entry points for each phase\n  types.ts\ndata/stripe/           structured ChangeEvent JSON, one file per detected entry\nexamples/sample-customer-repo/  synthetic fixture repo used to validate the scanner/fixer\nnpm install\ncp .env.example .env\n```\n\nFill in `.env`\n\ndirectly in your editor — never paste secrets into chat. `ANTHROPIC_API_KEY`\n\nis\nneeded for Phases 2 and 4; the `GITHUB_APP_*`\n\nvars are needed for Phase 5 (see below).\n\nNote onon Windows, npm's`npm run <script> -- --flags`\n\n:`.cmd`\n\nshim can silently drop`--flag`\n\n-style arguments passed after`--`\n\n. Every command below uses`npx tsx`\n\ndirectly instead, which doesn't have this problem — pass`--env-file=.env`\n\nif the command needs your API key.\n\n```\nnpx tsx src/cli.ts stripe --track acacia\n```\n\nFlags: `--track <name>`\n\n(default `acacia`\n\n; current live track is `dahlia`\n\n), `--all`\n\n(include\nnon-breaking entries), `--limit N`\n\n. Output: one JSON file per `ChangeEvent`\n\nunder `data/stripe/`\n\n.\n\nStripe serves clean raw markdown for any doc page via a `.md`\n\nsuffix — no HTML scraping. The index\npage flags each entry `Breaking`\n\n/`Non-breaking`\n\n; detail pages include a per-language parameter\ntable (`Added`\n\n/`Removed`\n\n) that Phase 1 parses deterministically.\n\n```\nnpx tsx --env-file=.env src/cli.ts stripe --track acacia --interpret\n```\n\nAttaches an `interpreted`\n\nfield to each `ChangeEvent`\n\n: an array of atomic changes\n(`param_renamed`\n\n, `param_removed`\n\n, `method_removed`\n\n, `response_shape_changed`\n\n, etc., each with\n`from`\n\n/`to`\n\n/`description`\n\n), a migration summary, and a confidence level. Deduplicates to one\ncanonical entry per target — validated against 10 real breaking changes across 3 release tracks.\nStripe's own tables are sometimes incomplete (e.g. list only the *new* values with no explicit\nmapping from old ones); the interpreter recovers the full mapping from prose in those cases.\n\n```\nnpx tsx src/scan.ts <path-to-customer-repo> --events <eventId1>,<eventId2>\n```\n\nOmit `--events`\n\nto scan against every interpreted event in `data/stripe/`\n\n. TypeScript/JavaScript\nonly. Uses `ts-morph`\n\nto parse the real AST rather than grep:\n\n**Call-argument matches**(request params) are scoped by tracing the call's callee chain back to a known Stripe client variable — either constructed locally (`new Stripe(...)`\n\n) or imported from another local module that constructs one (`import stripe from \"./server\"`\n\n, resolved recursively through re-exports via`ts-morph`\n\n's`getExportedDeclarations()`\n\n). Not just \"any object literal in a file that happens to import stripe.\"**Both call-argument and property-access matches** require the target's parent path segment (when it has one — e.g.`billing_mode`\n\nfor`billing_mode.type`\n\n,`errors`\n\nfor`Account.requirements.errors[].code`\n\n) to appear in the surrounding context — the receiver expression's text for property access, the enclosing call's callee text for call arguments — guarding generic leaf names like`type`\n\n,`status`\n\n, or`code`\n\nfrom matching unrelated objects.\n\nBoth are AST/text heuristics, not full type-checker provenance — meaningfully better than grep, but not infallible. Validated in two stages:\n\n- Against\n[examples/sample-customer-repo](/rtsdque/sentinel/blob/main/examples/sample-customer-repo), a fixture with deliberate false-positive traps (an unrelated local field sharing a param name, an unrelated Stripe`Event.type`\n\nread) — zero false positives. - Against a real, unmodified production codebase —\n[Cal.com](https://github.com/calcom/cal.com), a large TypeScript monorepo — which immediately surfaced a real gap the fixture couldn't: Cal.com constructs its Stripe client once in a shared module and imports it everywhere else, a very common production pattern our same-file-only detection completely missed (a false*negative*, confirmed empirically with a synthetic target before fixing). Fixing that then widened the surface for a second real false positive — a generic leaf name (`code`\n\n) matching an unrelated OAuth call — which is what the context-guard extension above addresses. Re-verified clean on both the fixture and Cal.com after both fixes. - Against a second real codebase —\n[Documenso](https://github.com/documenso/documenso), an npm workspace monorepo — which surfaced two more real findings:- The shared Stripe client is imported via a\n**monorepo package alias**(`@documenso/lib/server-only/stripe`\n\n), not a relative path.`ts-morph`\n\n's module resolution needs the workspace's`node_modules`\n\nsymlinks on disk to follow this — a bare`git clone`\n\nisn't enough;**dependencies must be installed (**, or every cross-package import silently resolves to nothing.`npm install`\n\n, ideally with`--ignore-scripts`\n\nfor read-only static analysis of a repo you don't otherwise trust) before scanning a workspace monorepo - Once installed, scanning surfaced a case the context guard can't help with: a\n**bare** rename target with no path segments at all (e.g.`name`\n\n, or in a live run,`filter`\n\n) has no context to check, so property-access matching falls back to \"anywhere in a file that touches a Stripe client\" — which matched an unrelated Prisma call's`.name`\n\nfield in testing, and in a real run against the 10 validated events, matched`Array.prototype.filter()`\n\ncalls (`.data.filter(...)`\n\n) against the Billing Alerts`filter`\n\nfield purely on name collision. Rather than chase a fragile heuristic fix (properly solving this needs type-checker provenance — does the receiver actually originate from a Stripe response? — which is a bigger lift than this pass), matches like this now carry`lowConfidence: true`\n\nand print with an explicit warning in`scan.ts`\n\n/`fix.ts`\n\noutput, so they're surfaced for extra scrutiny instead of looking identical to a well-scoped match.\n\n- The shared Stripe client is imported via a\n\n```\nnpx tsx --env-file=.env src/fix.ts <path-to-customer-repo> --events <eventId1>,<eventId2>\n```\n\nFor each scan match, sends the enclosing statement + the atomic change details to\n`claude-opus-4-8`\n\nand gets back a structured verdict: `canGenerateFix`\n\n, `confidence`\n\n, a proposed\nreplacement (or `null`\n\n), an explanation, and caveats. Prints two sections: fixes with a proposed\ndiff, and matches flagged for manual review with no diff proposed.\n\nThe generator is instructed to refuse rather than guess when a change alters a value's *shape*, not\njust its key name (e.g. a boolean becoming a string enum) — validated against exactly that trap and\ntwo others; it correctly declined all three with specific, useful reasoning, and correctly\ngenerated an accurate diff for a case that was genuinely safe (a pure structural rename with no\nvalue-type change).\n\n```\nnpx tsx --env-file=.env src/open-pr.ts <path-to-local-clone> --owner <org> --repo <repo> --base main --events <eventId>\n```\n\nRuns scan → fix → apply (with statement-level deduplication when multiple atomic changes land in\nthe same statement, and a Prettier pass to clean up indentation from spliced-in replacement text)\n→ opens one PR per `ChangeEvent`\n\n. The PR body always includes both the applied changes *and* the\nitems flagged for manual review — nothing is silently dropped — plus a disclaimer that nothing has\nbeen tested.\n\nThis touches customer code, so the app is scoped as narrowly as possible:\n\n| Permission | Level | Why |\n|---|---|---|\n| Contents | Read & write | Create a branch, commit the fix |\n| Pull requests | Read & write | Open the PR |\n| Metadata | Read (mandatory default) | Required by every GitHub App |\n\nNo `Administration`\n\n, no `Actions`\n\n, no org-wide access. No webhook subscription needed — this runs\non-demand from the CLI, not as an event listener, so you can leave \"Webhook\" unchecked when\nregistering the app.\n\n- Go to\n**github.com/settings/apps/new**(or your org's`/settings/apps/new`\n\nfor an org-owned app). - Set the permissions above under \"Repository permissions.\"\n- Uncheck \"Active\" under Webhook (not needed for this phase).\n- Under \"Where can this GitHub App be installed?\", choose \"Only on this account\" unless you specifically want it public.\n- Create the app, note the\n**App ID** shown on its settings page. - Scroll to \"Private keys\" →\n**Generate a private key**— downloads a`.pem`\n\nfile. Save it somewhere local (not inside this repo), and set`GITHUB_APP_PRIVATE_KEY_PATH`\n\nin`.env`\n\nto that file's path. Never paste the key's contents into chat or commit it anywhere. - Click\n**Install App**, choose a repo you own to test against (ideally a throwaway/test repo, not anything real, for the first run), and select \"Only select repositories.\" - After installing, the URL bar shows\n`.../installations/<number>`\n\n— that number is`GITHUB_APP_INSTALLATION_ID`\n\n. - Set\n`GITHUB_APP_ID`\n\nand`GITHUB_APP_INSTALLATION_ID`\n\nin`.env`\n\n.\n\nI can't do any of this myself — it requires your GitHub account and creates a real, persistent app\nregistration. Once it's set up, `open-pr.ts`\n\nonly ever acts within that installation's granted\nrepos.\n\nPostgres, via Drizzle ORM. Two ways to get one:\n\n**Cloud (needed for real scheduling)**: a free[Neon](https://neon.tech)or Supabase project — required if you actually want`.github/workflows/scheduled-scan.yml`\n\nto run unattended, since a GitHub Actions runner needs a database it can reach over the network, not one on your laptop.**Local, for just trying it out**:`docker compose up -d`\n\nstarts Postgres on`localhost:55432`\n\n(deliberately not the default 5432 — many machines already have a local Postgres service bound to that port, and connections would silently land on the wrong server with a confusing auth error instead of an obvious \"port in use\"). Then set`DATABASE_URL=postgresql://sentinel:sentinel@localhost:55432/sentinel`\n\nin`.env`\n\n. Fine for running`run-scheduled.ts`\n\nby hand; won't work for the cloud cron workflow.\n\nThree tables ([src/db/schema.ts](/rtsdque/sentinel/blob/main/src/db/schema.ts)):\n\n`change_events`\n\n— every detected entry, keyed by the same stable id the watcher already computes. Row existence is what \"have I seen this before\" means.`watched_repos`\n\n— registry of which repos to scan for which vendor, replacing manually passing`--owner`\n\n/`--repo`\n\non the CLI.`pull_requests`\n\n— log of PRs Sentinel has opened; prevents re-opening a PR for a (change event, repo) pair already handled, and is the source of truth for the \"users\" metric (distinct repos with at least one row here).\n\n```\nnpm run db:generate   # generate a migration from the schema\nnpm run db:migrate    # apply it\n\nnpx tsx src/backfill.ts <track>   # mark existing history as known WITHOUT interpreting or acting\n                                    # on any of it — run this once per track when first adopting\n                                    # Sentinel, so a scheduled run only reacts to changes from this\n                                    # point forward, not the entire historical backlog. No LLM calls.\n\nnode --env-file=.env --import tsx src/manage-repos.ts add --owner <o> --repo <r>\nnode --env-file=.env --import tsx src/manage-repos.ts list\n\nnode --env-file=.env --import tsx src/run-scheduled.ts --track <track>\n```\n\n`run-scheduled.ts`\n\nis the actual orchestrator: check a track for entries not yet in\n`change_events`\n\n→ interpret only the new ones (the cost-saving point of persistence — a backfilled\ntrack costs nothing to re-check) → for each active watched repo, clone it fresh via the GitHub\nApp's own installation token (never a persisted local path — a scheduled run has no guarantee of\nrunning on the same machine twice, and a cloud runner has no access to anyone's local filesystem at\nall), `npm install --ignore-scripts`\n\n(needed for monorepo package-alias imports to resolve — see\nPhase 3), scan, fix, apply, and open a PR — skipping any (event, repo) pair already in\n`pull_requests`\n\n.\n\n**This now genuinely runs on a schedule**, not just on-demand: [.github/workflows/scheduled-scan.yml](/rtsdque/sentinel/blob/main/.github/workflows/scheduled-scan.yml)\nruns it every 6 hours via GitHub Actions cron (plus a manual `workflow_dispatch`\n\ntrigger for\ntesting), reading secrets from the Actions secrets store rather than a local `.env`\n\n. Secret names\ncan't start with `GITHUB_`\n\n(a hard GitHub restriction), so they're stored as `SENTINEL_*`\n\nand mapped\nto the expected env var names inside the workflow's `env:`\n\nblock. The private key is provided as\nraw content via `GITHUB_APP_PRIVATE_KEY`\n\nin CI (no local file to point a path at); local dev keeps\nusing `GITHUB_APP_PRIVATE_KEY_PATH`\n\n— `loadPrivateKeyContent()`\n\nin\n[src/github/app.ts](/rtsdque/sentinel/blob/main/src/github/app.ts) checks the inline content first, falls back to the path.\n\n**Validated live in both places, not just typechecked:**\n\n- Locally: backfilled the real\n`basil`\n\ntrack (33 entries via deterministic parsing, zero LLM cost), removed one known-good entry to simulate it being newly discovered, ran the orchestrator against the real`rtsdque/app-test`\n\nrepo — it correctly found exactly the 1 new entry (not the other 32), interpreted only that one, cloned the repo fresh, opened a correct PR, and recorded it. Running it again immediately found 0 new entries and did nothing, confirming the dedup path. - In the cloud: pushed to a real (private) GitHub repo,\n[rtsdque/sentinel](https://github.com/rtsdque/sentinel), set the 5 required secrets, and triggered the workflow for real via`workflow_dispatch`\n\n— it connected to Neon from GitHub's own infrastructure, correctly found 0 new entries on`basil`\n\n(already fully processed), and exited clean. Confirms the whole chain — secrets,`npm ci`\n\n, DB connectivity, cloning — works outside this machine, not just here. - Multi-repo: registered a second fixture,\n[examples/sample-monorepo-repo](/rtsdque/sentinel/blob/main/examples/sample-monorepo-repo)(pushed to its own repo,`rtsdque/sentinel-test-monorepo`\n\n) — a real npm workspace monorepo with the Stripe client constructed in one package and imported by another via a package alias (`@fixture/stripe-client`\n\n), the exact shape found in Documenso. Until this test, that pattern had only ever been scanned read-only, never run through fix generation and PR creation. Re-ran the same balance-settings change against both watched repos in a single orchestrator invocation — it correctly opened two independent, correctly-scoped PRs (`app-test#5`\n\n,`sentinel-test-monorepo#1`\n\n), each with the right diff, both logged against the right repo id in`pull_requests`\n\n.\n\n**Known gap surfaced by this design, not yet resolved**: a watched repo is only checked against\nevents discovered *after* it was registered. Adding a new repo to `watched_repos`\n\ndoesn't scan it\nagainst events already in `change_events`\n\n— there's no \"backfill this repo against the last N\nknown changes\" path yet. Fine for the steady-state case (repos stay registered, watcher runs\ncontinuously), but a gap for onboarding a repo after some changes have already been seen.\n\n**This repo's own visibility**:[rtsdque/sentinel](https://github.com/rtsdque/sentinel)is currently private. Going public is a deliberate later decision, not an accident to guard against — but worth knowing the code,`data/stripe/*.json`\n\n, and migration history are all sitting in a repo that isn't public yet, so nothing here has had public scrutiny.**Scanner/fixer scope**: TypeScript/JavaScript + Stripe only. No Python, no other vendors yet.** AST heuristics, not type-checker provenance**: both the scanner's context guards and the fixer's statement-boundary matching are heuristics that can be fooled by unusual code shapes. Treat every match and diff as a candidate for review, not proof. Cross-file client resolution handles direct exports, re-export chains, and monorepo package-alias imports (confirmed against both Cal.com and Documenso) — but only once the repo's dependencies are actually installed (see Phase 3 above); a bare`git clone`\n\nisn't enough for workspace monorepos. Still untested: computed member access (`stripe[\"oauth\"][\"token\"]`\n\n), a client factory function instead of a bare exported instance. Bare (path-less) rename targets with a common English-word leaf (`name`\n\n,`filter`\n\n,`id`\n\n) remain a known false-positive risk for property-access matching — surfaced via`lowConfidence: true`\n\non the`ScanMatch`\n\n/`FixCandidate`\n\n, not eliminated; fully closing this needs type-checker provenance (does the receiver actually originate from a Stripe response?), which is future work.**One PR per file-touching event, sequential**:`open-pr.ts`\n\ndoesn't yet dedupe against a PR that already exists for the same branch beyond a clear error message — no update-existing-PR flow yet.", "url": "https://wpnews.pro/news/sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land", "canonical_source": "https://github.com/rtsdque/sentinel", "published_at": "2026-07-29 23:16:04+00:00", "updated_at": "2026-07-29 23:52:38.097138+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "large-language-models"], "entities": ["Sentinel", "Stripe", "Dependabot", "Renovate", "oasdiff", "openapi-diff", "Visualping", "Theneo"], "alternates": {"html": "https://wpnews.pro/news/sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land", "markdown": "https://wpnews.pro/news/sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land.md", "text": "https://wpnews.pro/news/sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land.txt", "jsonld": "https://wpnews.pro/news/sentinel-opens-a-pr-to-fix-stripe-breaking-changes-before-they-land.jsonld"}}