Sentinel – opens a PR to fix Stripe breaking changes before they land

Sentinel, a new open-source tool, detects breaking changes in vendor API changelogs before they affect customer codebases, scans repositories for affected usages, and opens a pull request with fix diffs before the sunset date. The tool currently supports Stripe and TypeScript/JavaScript repos, with phases 1–6 built, and requires only Node to detect a real breaking change without an account or API key.

Watches vendor API changelogs, detects breaking changes before they hit customer codebases, scans a customer repo for affected usages, generates reviewable fix diffs, and opens a PR — before the sunset date, not after something breaks in production. See it detect a real breaking change with no account, API key, or database — just Node: git clone https://github.com/rtsdque/sentinel.git cd sentinel npm install npx tsx src/cli.ts stripe --track acacia That's it — this scrapes Stripe's real changelog and prints a fully structured breaking-change event to your terminal, deterministically no LLM call . Everything past this point interpreting changes with an LLM, scanning a codebase, opening PRs, running on a schedule layers on top and needs progressively more setup — jump to the phase you want below, or npm test to run the suite also needs nothing but Node . This is a real diff Sentinel generated and opened as a PR, unedited, against a test repo — Stripe nested three Balance Settings request fields under a new required payments object https://docs.stripe.com/changelog/basil/2025-08-27/balance-settings-nested-in-payments-field.md , and Sentinel found the affected call site and proposed the migration on its own: export async function enableNegativeBalanceDebits { return stripe.balanceSettings.update { - debit negative balances: true, - payouts: { - schedule: { interval: "daily" }, + payments: { + debit negative balances: true, + payouts: { + schedule: { interval: "daily" }, + }, }, } ; } The PR body links the upstream changelog entry, states the deadline if there is one, and separately lists anything it couldn't confidently fix — see Phase 4 phase-4--fix-generator and Phase 5 phase-5--pr-bot below for what makes it refuse rather than guess. Reasonable question — here's what's actually different, based on a real look at the landscape, not a guess: | Tool | What it does | Where it stops | |---|---|---| Dependabot incl. AI-agent remediation | Reacts to a known CVE or a new package version in your manifest | Anchored to your dependency file — never reads a vendor's changelog prose, only version diffs | Renovate | Same, plus customDatasources for arbitrary version feeds | Still just version tracking — no interpretation of what changed or why it breaks you | oasdiff / openapi-diff | Diffs two OpenAPI specs, flags breaking changes precisely | Needs the vendor's own spec-versioned CI — doesn't touch your codebase at all | Visualping / Theneo / PageCrawl | Watch a changelog page, alert on any change | Stops at a notification. You still read it, find the usages, and fix them yourself | ChangeSpec | An open JSON format for vendors to publish structured change events | A spec waiting for vendor adoption — Stripe doesn't publish in it today, so someone still has to read the prose | Every one of these is genuinely good at one piece. None of them do the thing this YC RFS actually asked for: watch a vendor's unstructured changelog prose, turn it into a structured breaking change, find every place your specific codebase uses the affected surface, and propose the fix — before your build breaks, not after. That's the gap Sentinel targets, and as far as we could find during a real competitive pass not a hunch , nobody had shipped the full pipeline yet. Phases 1–6 built , scoped to Stripe + TypeScript/JavaScript customer repos only see project scope notes — deliberately not broad . | Phase | What it does | Entry point | |---|---|---| | 1. Change Watcher | Scrapes Stripe's changelog, deterministically parses breaking-change entries into structured ChangeEvent s | src/cli.ts | | 2. Change Interpreter | LLM step claude-opus-4-8 that decomposes prose + the parsed param table into atomic, machine-actionable changes | src/cli.ts --interpret | | 3. Codebase Scanner | AST-based ts-morph scan of a TS/JS repo for usages of the affected API surface, scoped to real Stripe client calls | src/scan.ts | | 4. Fix Generator | Per-match LLM call that proposes a diff or explicitly declines with a reason — never guesses | src/fix.ts | | 5. PR Bot | GitHub App that applies approved fixes, formats with Prettier, and opens a PR — every PR states what wasn't auto-fixed | src/open-pr.ts | | 6. Persistence & Scheduling | Postgres-backed "have I seen this" store + a repo registry, so a run only acts on genuinely new changes | src/run-scheduled.ts | Nothing in this pipeline auto-merges. Every fix is generated for human/PR review; the PR body explicitly separates "proposed changes" from "needs manual review," because there's no customer test suite to verify correctness against. src/ lib/ fetch wrapper, markdown-table parsing watcher/stripe/ Phase 1: changelog fetch + parse + ChangeEvent builder interpreter/stripe.ts Phase 2: LLM decomposition into atomic changes scanner/typescript.ts Phase 3: ts-morph AST scan for affected usages fixer/stripe.ts Phase 4: per-match fix generation + apply/consolidate/format github/ app.ts GitHub App installation auth pr.ts branch + commit + PR creation cli.ts / scan.ts / fix.ts / open-pr.ts entry points for each phase types.ts data/stripe/ structured ChangeEvent JSON, one file per detected entry examples/sample-customer-repo/ synthetic fixture repo used to validate the scanner/fixer npm install cp .env.example .env Fill in .env directly in your editor — never paste secrets into chat. ANTHROPIC API KEY is needed for Phases 2 and 4; the GITHUB APP vars are needed for Phase 5 see below . Note onon Windows, npm's npm run <script -- --flags : .cmd shim can silently drop --flag -style arguments passed after -- . Every command below uses npx tsx directly instead, which doesn't have this problem — pass --env-file=.env if the command needs your API key. npx tsx src/cli.ts stripe --track acacia Flags: --track <name default acacia ; current live track is dahlia , --all include non-breaking entries , --limit N . Output: one JSON file per ChangeEvent under data/stripe/ . Stripe serves clean raw markdown for any doc page via a .md suffix — no HTML scraping. The index page flags each entry Breaking / Non-breaking ; detail pages include a per-language parameter table Added / Removed that Phase 1 parses deterministically. npx tsx --env-file=.env src/cli.ts stripe --track acacia --interpret Attaches an interpreted field to each ChangeEvent : an array of atomic changes param renamed , param removed , method removed , response shape changed , etc., each with from / to / description , a migration summary, and a confidence level. Deduplicates to one canonical entry per target — validated against 10 real breaking changes across 3 release tracks. Stripe's own tables are sometimes incomplete e.g. list only the new values with no explicit mapping from old ones ; the interpreter recovers the full mapping from prose in those cases. npx tsx src/scan.ts <path-to-customer-repo --events <eventId1 ,<eventId2 Omit --events to scan against every interpreted event in data/stripe/ . TypeScript/JavaScript only. Uses ts-morph to parse the real AST rather than grep: Call-argument matches request params are scoped by tracing the call's callee chain back to a known Stripe client variable — either constructed locally new Stripe ... or imported from another local module that constructs one import stripe from "./server" , resolved recursively through re-exports via ts-morph 's getExportedDeclarations . Not just "any object literal in a file that happens to import stripe." Both call-argument and property-access matches require the target's parent path segment when it has one — e.g. billing mode for billing mode.type , errors for Account.requirements.errors .code to appear in the surrounding context — the receiver expression's text for property access, the enclosing call's callee text for call arguments — guarding generic leaf names like type , status , or code from matching unrelated objects. Both are AST/text heuristics, not full type-checker provenance — meaningfully better than grep, but not infallible. Validated in two stages: - Against examples/sample-customer-repo /rtsdque/sentinel/blob/main/examples/sample-customer-repo , a fixture with deliberate false-positive traps an unrelated local field sharing a param name, an unrelated Stripe Event.type read — zero false positives. - Against a real, unmodified production codebase — Cal.com https://github.com/calcom/cal.com , a large TypeScript monorepo — which immediately surfaced a real gap the fixture couldn't: Cal.com constructs its Stripe client once in a shared module and imports it everywhere else, a very common production pattern our same-file-only detection completely missed a false negative , confirmed empirically with a synthetic target before fixing . Fixing that then widened the surface for a second real false positive — a generic leaf name code matching an unrelated OAuth call — which is what the context-guard extension above addresses. Re-verified clean on both the fixture and Cal.com after both fixes. - Against a second real codebase — Documenso https://github.com/documenso/documenso , an npm workspace monorepo — which surfaced two more real findings:- The shared Stripe client is imported via a monorepo package alias @documenso/lib/server-only/stripe , not a relative path. ts-morph 's module resolution needs the workspace's node modules symlinks on disk to follow this — a bare git clone isn't enough; dependencies must be installed , or every cross-package import silently resolves to nothing. npm install , ideally with --ignore-scripts for read-only static analysis of a repo you don't otherwise trust before scanning a workspace monorepo - Once installed, scanning surfaced a case the context guard can't help with: a bare rename target with no path segments at all e.g. name , or in a live run, filter has no context to check, so property-access matching falls back to "anywhere in a file that touches a Stripe client" — which matched an unrelated Prisma call's .name field in testing, and in a real run against the 10 validated events, matched Array.prototype.filter calls .data.filter ... against the Billing Alerts filter field purely on name collision. Rather than chase a fragile heuristic fix properly solving this needs type-checker provenance — does the receiver actually originate from a Stripe response? — which is a bigger lift than this pass , matches like this now carry lowConfidence: true and print with an explicit warning in scan.ts / fix.ts output, so they're surfaced for extra scrutiny instead of looking identical to a well-scoped match. - The shared Stripe client is imported via a npx tsx --env-file=.env src/fix.ts <path-to-customer-repo --events <eventId1 ,<eventId2 For each scan match, sends the enclosing statement + the atomic change details to claude-opus-4-8 and gets back a structured verdict: canGenerateFix , confidence , a proposed replacement or null , an explanation, and caveats. Prints two sections: fixes with a proposed diff, and matches flagged for manual review with no diff proposed. The generator is instructed to refuse rather than guess when a change alters a value's shape , not just its key name e.g. a boolean becoming a string enum — validated against exactly that trap and two others; it correctly declined all three with specific, useful reasoning, and correctly generated an accurate diff for a case that was genuinely safe a pure structural rename with no value-type change . npx tsx --env-file=.env src/open-pr.ts <path-to-local-clone --owner <org --repo <repo --base main --events <eventId Runs scan → fix → apply with statement-level deduplication when multiple atomic changes land in the same statement, and a Prettier pass to clean up indentation from spliced-in replacement text → opens one PR per ChangeEvent . The PR body always includes both the applied changes and the items flagged for manual review — nothing is silently dropped — plus a disclaimer that nothing has been tested. This touches customer code, so the app is scoped as narrowly as possible: | Permission | Level | Why | |---|---|---| | Contents | Read & write | Create a branch, commit the fix | | Pull requests | Read & write | Open the PR | | Metadata | Read mandatory default | Required by every GitHub App | No Administration , no Actions , no org-wide access. No webhook subscription needed — this runs on-demand from the CLI, not as an event listener, so you can leave "Webhook" unchecked when registering the app. - Go to github.com/settings/apps/new or your org's /settings/apps/new for an org-owned app . - Set the permissions above under "Repository permissions." - Uncheck "Active" under Webhook not needed for this phase . - Under "Where can this GitHub App be installed?", choose "Only on this account" unless you specifically want it public. - Create the app, note the App ID shown on its settings page. - Scroll to "Private keys" → Generate a private key — downloads a .pem file. Save it somewhere local not inside this repo , and set GITHUB APP PRIVATE KEY PATH in .env to that file's path. Never paste the key's contents into chat or commit it anywhere. - Click Install App , choose a repo you own to test against ideally a throwaway/test repo, not anything real, for the first run , and select "Only select repositories." - After installing, the URL bar shows .../installations/<number — that number is GITHUB APP INSTALLATION ID . - Set GITHUB APP ID and GITHUB APP INSTALLATION ID in .env . I can't do any of this myself — it requires your GitHub account and creates a real, persistent app registration. Once it's set up, open-pr.ts only ever acts within that installation's granted repos. Postgres, via Drizzle ORM. Two ways to get one: Cloud needed for real scheduling : a free Neon https://neon.tech or Supabase project — required if you actually want .github/workflows/scheduled-scan.yml to run unattended, since a GitHub Actions runner needs a database it can reach over the network, not one on your laptop. Local, for just trying it out : docker compose up -d starts Postgres on localhost:55432 deliberately not the default 5432 — many machines already have a local Postgres service bound to that port, and connections would silently land on the wrong server with a confusing auth error instead of an obvious "port in use" . Then set DATABASE URL=postgresql://sentinel:sentinel@localhost:55432/sentinel in .env . Fine for running run-scheduled.ts by hand; won't work for the cloud cron workflow. Three tables src/db/schema.ts /rtsdque/sentinel/blob/main/src/db/schema.ts : change events — every detected entry, keyed by the same stable id the watcher already computes. Row existence is what "have I seen this before" means. watched repos — registry of which repos to scan for which vendor, replacing manually passing --owner / --repo on the CLI. pull requests — log of PRs Sentinel has opened; prevents re-opening a PR for a change event, repo pair already handled, and is the source of truth for the "users" metric distinct repos with at least one row here . npm run db:generate generate a migration from the schema npm run db:migrate apply it npx tsx src/backfill.ts <track mark existing history as known WITHOUT interpreting or acting on any of it — run this once per track when first adopting Sentinel, so a scheduled run only reacts to changes from this point forward, not the entire historical backlog. No LLM calls. node --env-file=.env --import tsx src/manage-repos.ts add --owner <o --repo <r node --env-file=.env --import tsx src/manage-repos.ts list node --env-file=.env --import tsx src/run-scheduled.ts --track <track run-scheduled.ts is the actual orchestrator: check a track for entries not yet in change events → interpret only the new ones the cost-saving point of persistence — a backfilled track costs nothing to re-check → for each active watched repo, clone it fresh via the GitHub App's own installation token never a persisted local path — a scheduled run has no guarantee of running on the same machine twice, and a cloud runner has no access to anyone's local filesystem at all , npm install --ignore-scripts needed for monorepo package-alias imports to resolve — see Phase 3 , scan, fix, apply, and open a PR — skipping any event, repo pair already in pull requests . This now genuinely runs on a schedule , not just on-demand: .github/workflows/scheduled-scan.yml /rtsdque/sentinel/blob/main/.github/workflows/scheduled-scan.yml runs it every 6 hours via GitHub Actions cron plus a manual workflow dispatch trigger for testing , reading secrets from the Actions secrets store rather than a local .env . Secret names can't start with GITHUB a hard GitHub restriction , so they're stored as SENTINEL and mapped to the expected env var names inside the workflow's env: block. The private key is provided as raw content via GITHUB APP PRIVATE KEY in CI no local file to point a path at ; local dev keeps using GITHUB APP PRIVATE KEY PATH — loadPrivateKeyContent in src/github/app.ts /rtsdque/sentinel/blob/main/src/github/app.ts checks the inline content first, falls back to the path. Validated live in both places, not just typechecked: - Locally: backfilled the real basil track 33 entries via deterministic parsing, zero LLM cost , removed one known-good entry to simulate it being newly discovered, ran the orchestrator against the real rtsdque/app-test repo — it correctly found exactly the 1 new entry not the other 32 , interpreted only that one, cloned the repo fresh, opened a correct PR, and recorded it. Running it again immediately found 0 new entries and did nothing, confirming the dedup path. - In the cloud: pushed to a real private GitHub repo, rtsdque/sentinel https://github.com/rtsdque/sentinel , set the 5 required secrets, and triggered the workflow for real via workflow dispatch — it connected to Neon from GitHub's own infrastructure, correctly found 0 new entries on basil already fully processed , and exited clean. Confirms the whole chain — secrets, npm ci , DB connectivity, cloning — works outside this machine, not just here. - Multi-repo: registered a second fixture, examples/sample-monorepo-repo /rtsdque/sentinel/blob/main/examples/sample-monorepo-repo pushed to its own repo, rtsdque/sentinel-test-monorepo — a real npm workspace monorepo with the Stripe client constructed in one package and imported by another via a package alias @fixture/stripe-client , the exact shape found in Documenso. Until this test, that pattern had only ever been scanned read-only, never run through fix generation and PR creation. Re-ran the same balance-settings change against both watched repos in a single orchestrator invocation — it correctly opened two independent, correctly-scoped PRs app-test 5 , sentinel-test-monorepo 1 , each with the right diff, both logged against the right repo id in pull requests . Known gap surfaced by this design, not yet resolved : a watched repo is only checked against events discovered after it was registered. Adding a new repo to watched repos doesn't scan it against events already in change events — there's no "backfill this repo against the last N known changes" path yet. Fine for the steady-state case repos stay registered, watcher runs continuously , but a gap for onboarding a repo after some changes have already been seen. This repo's own visibility : rtsdque/sentinel https://github.com/rtsdque/sentinel is currently private. Going public is a deliberate later decision, not an accident to guard against — but worth knowing the code, data/stripe/ .json , and migration history are all sitting in a repo that isn't public yet, so nothing here has had public scrutiny. Scanner/fixer scope : TypeScript/JavaScript + Stripe only. No Python, no other vendors yet. AST heuristics, not type-checker provenance : both the scanner's context guards and the fixer's statement-boundary matching are heuristics that can be fooled by unusual code shapes. Treat every match and diff as a candidate for review, not proof. Cross-file client resolution handles direct exports, re-export chains, and monorepo package-alias imports confirmed against both Cal.com and Documenso — but only once the repo's dependencies are actually installed see Phase 3 above ; a bare git clone isn't enough for workspace monorepos. Still untested: computed member access stripe "oauth" "token" , a client factory function instead of a bare exported instance. Bare path-less rename targets with a common English-word leaf name , filter , id remain a known false-positive risk for property-access matching — surfaced via lowConfidence: true on the ScanMatch / FixCandidate , not eliminated; fully closing this needs type-checker provenance does the receiver actually originate from a Stripe response? , which is future work. One PR per file-touching event, sequential : open-pr.ts doesn't yet dedupe against a PR that already exists for the same branch beyond a clear error message — no update-existing-PR flow yet.