# Sentinel – opens a PR to fix Stripe breaking changes before they land

> Source: <https://github.com/rtsdque/sentinel>
> Published: 2026-07-29 23:16:04+00:00

Watches vendor API changelogs, detects breaking changes before they hit customer codebases, scans a customer repo for affected usages, generates reviewable fix diffs, and opens a PR — before the sunset date, not after something breaks in production.

See it detect a real breaking change with no account, API key, or database — just Node:

```
git clone https://github.com/rtsdque/sentinel.git
cd sentinel
npm install
npx tsx src/cli.ts stripe --track acacia
```

That's it — this scrapes Stripe's real changelog and prints a fully structured breaking-change
event to your terminal, deterministically (no LLM call). Everything past this point (interpreting
changes with an LLM, scanning a codebase, opening PRs, running on a schedule) layers on top and
needs progressively more setup — jump to the phase you want below, or `npm test`

to run the
suite (also needs nothing but Node).

This is a real diff Sentinel generated and opened as a PR, unedited, against a test repo — Stripe
[nested three Balance Settings request fields under a new required payments object](https://docs.stripe.com/changelog/basil/2025-08-27/balance-settings-nested-in-payments-field.md),
and Sentinel found the affected call site and proposed the migration on its own:

```
 export async function enableNegativeBalanceDebits() {
   return stripe.balanceSettings.update({
-    debit_negative_balances: true,
-    payouts: {
-      schedule: { interval: "daily" },
+    payments: {
+      debit_negative_balances: true,
+      payouts: {
+        schedule: { interval: "daily" },
+      },
     },
   });
 }
```

The PR body links the upstream changelog entry, states the deadline if there is one, and separately
lists anything it *couldn't* confidently fix — see [Phase 4](#phase-4--fix-generator) and
[Phase 5](#phase-5--pr-bot) below for what makes it refuse rather than guess.

Reasonable question — here's what's actually different, based on a real look at the landscape, not a guess:

| Tool | What it does | Where it stops |
|---|---|---|
Dependabot (incl. AI-agent remediation) |
Reacts to a known CVE or a new package version in your manifest | Anchored to your dependency file — never reads a vendor's changelog prose, only version diffs |
Renovate |
Same, plus `customDatasources` for arbitrary version feeds |
Still just version tracking — no interpretation of what changed or why it breaks you |
oasdiff / openapi-diff |
Diffs two OpenAPI specs, flags breaking changes precisely | Needs the vendor's own spec-versioned CI — doesn't touch your codebase at all |
Visualping / Theneo / PageCrawl |
Watch a changelog page, alert on any change | Stops at a notification. You still read it, find the usages, and fix them yourself |
ChangeSpec |
An open JSON format for vendors to publish structured change events | A spec waiting for vendor adoption — Stripe doesn't publish in it today, so someone still has to read the prose |

Every one of these is genuinely good at one piece. None of them do the thing this YC RFS actually
asked for: watch a vendor's *unstructured* changelog prose, turn it into a structured breaking
change, find every place *your specific codebase* uses the affected surface, and propose the fix —
before your build breaks, not after. That's the gap Sentinel targets, and as far as we could find
during a real competitive pass (not a hunch), nobody had shipped the full pipeline yet.

**Phases 1–6 built**, scoped to Stripe + TypeScript/JavaScript customer repos only (see project
scope notes — deliberately not broad).

| Phase | What it does | Entry point |
|---|---|---|
| 1. Change Watcher | Scrapes Stripe's changelog, deterministically parses breaking-change entries into structured `ChangeEvent` s |
`src/cli.ts` |
| 2. Change Interpreter | LLM step (`claude-opus-4-8` ) that decomposes prose + the parsed param table into atomic, machine-actionable changes |
`src/cli.ts --interpret` |
| 3. Codebase Scanner | AST-based (`ts-morph` ) scan of a TS/JS repo for usages of the affected API surface, scoped to real Stripe client calls |
`src/scan.ts` |
| 4. Fix Generator | Per-match LLM call that proposes a diff or explicitly declines with a reason — never guesses | `src/fix.ts` |
| 5. PR Bot | GitHub App that applies approved fixes, formats with Prettier, and opens a PR — every PR states what wasn't auto-fixed | `src/open-pr.ts` |
| 6. Persistence & Scheduling | Postgres-backed "have I seen this" store + a repo registry, so a run only acts on genuinely new changes | `src/run-scheduled.ts` |

Nothing in this pipeline auto-merges. Every fix is generated for human/PR review; the PR body explicitly separates "proposed changes" from "needs manual review," because there's no customer test suite to verify correctness against.

```
src/
  lib/                 fetch wrapper, markdown-table parsing
  watcher/stripe/       Phase 1: changelog fetch + parse + ChangeEvent builder
  interpreter/stripe.ts Phase 2: LLM decomposition into atomic changes
  scanner/typescript.ts Phase 3: ts-morph AST scan for affected usages
  fixer/stripe.ts       Phase 4: per-match fix generation + apply/consolidate/format
  github/
    app.ts               GitHub App installation auth
    pr.ts                 branch + commit + PR creation
  cli.ts / scan.ts / fix.ts / open-pr.ts   entry points for each phase
  types.ts
data/stripe/           structured ChangeEvent JSON, one file per detected entry
examples/sample-customer-repo/  synthetic fixture repo used to validate the scanner/fixer
npm install
cp .env.example .env
```

Fill in `.env`

directly in your editor — never paste secrets into chat. `ANTHROPIC_API_KEY`

is
needed for Phases 2 and 4; the `GITHUB_APP_*`

vars are needed for Phase 5 (see below).

Note onon Windows, npm's`npm run <script> -- --flags`

:`.cmd`

shim can silently drop`--flag`

-style arguments passed after`--`

. Every command below uses`npx tsx`

directly instead, which doesn't have this problem — pass`--env-file=.env`

if the command needs your API key.

```
npx tsx src/cli.ts stripe --track acacia
```

Flags: `--track <name>`

(default `acacia`

; current live track is `dahlia`

), `--all`

(include
non-breaking entries), `--limit N`

. Output: one JSON file per `ChangeEvent`

under `data/stripe/`

.

Stripe serves clean raw markdown for any doc page via a `.md`

suffix — no HTML scraping. The index
page flags each entry `Breaking`

/`Non-breaking`

; detail pages include a per-language parameter
table (`Added`

/`Removed`

) that Phase 1 parses deterministically.

```
npx tsx --env-file=.env src/cli.ts stripe --track acacia --interpret
```

Attaches an `interpreted`

field to each `ChangeEvent`

: an array of atomic changes
(`param_renamed`

, `param_removed`

, `method_removed`

, `response_shape_changed`

, etc., each with
`from`

/`to`

/`description`

), a migration summary, and a confidence level. Deduplicates to one
canonical entry per target — validated against 10 real breaking changes across 3 release tracks.
Stripe's own tables are sometimes incomplete (e.g. list only the *new* values with no explicit
mapping from old ones); the interpreter recovers the full mapping from prose in those cases.

```
npx tsx src/scan.ts <path-to-customer-repo> --events <eventId1>,<eventId2>
```

Omit `--events`

to scan against every interpreted event in `data/stripe/`

. TypeScript/JavaScript
only. Uses `ts-morph`

to parse the real AST rather than grep:

**Call-argument matches**(request params) are scoped by tracing the call's callee chain back to a known Stripe client variable — either constructed locally (`new Stripe(...)`

) or imported from another local module that constructs one (`import stripe from "./server"`

, resolved recursively through re-exports via`ts-morph`

's`getExportedDeclarations()`

). Not just "any object literal in a file that happens to import stripe."**Both call-argument and property-access matches** require the target's parent path segment (when it has one — e.g.`billing_mode`

for`billing_mode.type`

,`errors`

for`Account.requirements.errors[].code`

) to appear in the surrounding context — the receiver expression's text for property access, the enclosing call's callee text for call arguments — guarding generic leaf names like`type`

,`status`

, or`code`

from matching unrelated objects.

Both are AST/text heuristics, not full type-checker provenance — meaningfully better than grep, but not infallible. Validated in two stages:

- Against
[examples/sample-customer-repo](/rtsdque/sentinel/blob/main/examples/sample-customer-repo), a fixture with deliberate false-positive traps (an unrelated local field sharing a param name, an unrelated Stripe`Event.type`

read) — zero false positives. - Against a real, unmodified production codebase —
[Cal.com](https://github.com/calcom/cal.com), a large TypeScript monorepo — which immediately surfaced a real gap the fixture couldn't: Cal.com constructs its Stripe client once in a shared module and imports it everywhere else, a very common production pattern our same-file-only detection completely missed (a false*negative*, confirmed empirically with a synthetic target before fixing). Fixing that then widened the surface for a second real false positive — a generic leaf name (`code`

) matching an unrelated OAuth call — which is what the context-guard extension above addresses. Re-verified clean on both the fixture and Cal.com after both fixes. - Against a second real codebase —
[Documenso](https://github.com/documenso/documenso), an npm workspace monorepo — which surfaced two more real findings:- The shared Stripe client is imported via a
**monorepo package alias**(`@documenso/lib/server-only/stripe`

), not a relative path.`ts-morph`

's module resolution needs the workspace's`node_modules`

symlinks on disk to follow this — a bare`git clone`

isn't enough;**dependencies must be installed (**, or every cross-package import silently resolves to nothing.`npm install`

, ideally with`--ignore-scripts`

for read-only static analysis of a repo you don't otherwise trust) before scanning a workspace monorepo - Once installed, scanning surfaced a case the context guard can't help with: a
**bare** rename target with no path segments at all (e.g.`name`

, or in a live run,`filter`

) has no context to check, so property-access matching falls back to "anywhere in a file that touches a Stripe client" — which matched an unrelated Prisma call's`.name`

field in testing, and in a real run against the 10 validated events, matched`Array.prototype.filter()`

calls (`.data.filter(...)`

) against the Billing Alerts`filter`

field purely on name collision. Rather than chase a fragile heuristic fix (properly solving this needs type-checker provenance — does the receiver actually originate from a Stripe response? — which is a bigger lift than this pass), matches like this now carry`lowConfidence: true`

and print with an explicit warning in`scan.ts`

/`fix.ts`

output, so they're surfaced for extra scrutiny instead of looking identical to a well-scoped match.

- The shared Stripe client is imported via a

```
npx tsx --env-file=.env src/fix.ts <path-to-customer-repo> --events <eventId1>,<eventId2>
```

For each scan match, sends the enclosing statement + the atomic change details to
`claude-opus-4-8`

and gets back a structured verdict: `canGenerateFix`

, `confidence`

, a proposed
replacement (or `null`

), an explanation, and caveats. Prints two sections: fixes with a proposed
diff, and matches flagged for manual review with no diff proposed.

The generator is instructed to refuse rather than guess when a change alters a value's *shape*, not
just its key name (e.g. a boolean becoming a string enum) — validated against exactly that trap and
two others; it correctly declined all three with specific, useful reasoning, and correctly
generated an accurate diff for a case that was genuinely safe (a pure structural rename with no
value-type change).

```
npx tsx --env-file=.env src/open-pr.ts <path-to-local-clone> --owner <org> --repo <repo> --base main --events <eventId>
```

Runs scan → fix → apply (with statement-level deduplication when multiple atomic changes land in
the same statement, and a Prettier pass to clean up indentation from spliced-in replacement text)
→ opens one PR per `ChangeEvent`

. The PR body always includes both the applied changes *and* the
items flagged for manual review — nothing is silently dropped — plus a disclaimer that nothing has
been tested.

This touches customer code, so the app is scoped as narrowly as possible:

| Permission | Level | Why |
|---|---|---|
| Contents | Read & write | Create a branch, commit the fix |
| Pull requests | Read & write | Open the PR |
| Metadata | Read (mandatory default) | Required by every GitHub App |

No `Administration`

, no `Actions`

, no org-wide access. No webhook subscription needed — this runs
on-demand from the CLI, not as an event listener, so you can leave "Webhook" unchecked when
registering the app.

- Go to
**github.com/settings/apps/new**(or your org's`/settings/apps/new`

for an org-owned app). - Set the permissions above under "Repository permissions."
- Uncheck "Active" under Webhook (not needed for this phase).
- Under "Where can this GitHub App be installed?", choose "Only on this account" unless you specifically want it public.
- Create the app, note the
**App ID** shown on its settings page. - Scroll to "Private keys" →
**Generate a private key**— downloads a`.pem`

file. Save it somewhere local (not inside this repo), and set`GITHUB_APP_PRIVATE_KEY_PATH`

in`.env`

to that file's path. Never paste the key's contents into chat or commit it anywhere. - Click
**Install App**, choose a repo you own to test against (ideally a throwaway/test repo, not anything real, for the first run), and select "Only select repositories." - After installing, the URL bar shows
`.../installations/<number>`

— that number is`GITHUB_APP_INSTALLATION_ID`

. - Set
`GITHUB_APP_ID`

and`GITHUB_APP_INSTALLATION_ID`

in`.env`

.

I can't do any of this myself — it requires your GitHub account and creates a real, persistent app
registration. Once it's set up, `open-pr.ts`

only ever acts within that installation's granted
repos.

Postgres, via Drizzle ORM. Two ways to get one:

**Cloud (needed for real scheduling)**: a free[Neon](https://neon.tech)or Supabase project — required if you actually want`.github/workflows/scheduled-scan.yml`

to run unattended, since a GitHub Actions runner needs a database it can reach over the network, not one on your laptop.**Local, for just trying it out**:`docker compose up -d`

starts Postgres on`localhost:55432`

(deliberately not the default 5432 — many machines already have a local Postgres service bound to that port, and connections would silently land on the wrong server with a confusing auth error instead of an obvious "port in use"). Then set`DATABASE_URL=postgresql://sentinel:sentinel@localhost:55432/sentinel`

in`.env`

. Fine for running`run-scheduled.ts`

by hand; won't work for the cloud cron workflow.

Three tables ([src/db/schema.ts](/rtsdque/sentinel/blob/main/src/db/schema.ts)):

`change_events`

— every detected entry, keyed by the same stable id the watcher already computes. Row existence is what "have I seen this before" means.`watched_repos`

— registry of which repos to scan for which vendor, replacing manually passing`--owner`

/`--repo`

on the CLI.`pull_requests`

— log of PRs Sentinel has opened; prevents re-opening a PR for a (change event, repo) pair already handled, and is the source of truth for the "users" metric (distinct repos with at least one row here).

```
npm run db:generate   # generate a migration from the schema
npm run db:migrate    # apply it

npx tsx src/backfill.ts <track>   # mark existing history as known WITHOUT interpreting or acting
                                    # on any of it — run this once per track when first adopting
                                    # Sentinel, so a scheduled run only reacts to changes from this
                                    # point forward, not the entire historical backlog. No LLM calls.

node --env-file=.env --import tsx src/manage-repos.ts add --owner <o> --repo <r>
node --env-file=.env --import tsx src/manage-repos.ts list

node --env-file=.env --import tsx src/run-scheduled.ts --track <track>
```

`run-scheduled.ts`

is the actual orchestrator: check a track for entries not yet in
`change_events`

→ interpret only the new ones (the cost-saving point of persistence — a backfilled
track costs nothing to re-check) → for each active watched repo, clone it fresh via the GitHub
App's own installation token (never a persisted local path — a scheduled run has no guarantee of
running on the same machine twice, and a cloud runner has no access to anyone's local filesystem at
all), `npm install --ignore-scripts`

(needed for monorepo package-alias imports to resolve — see
Phase 3), scan, fix, apply, and open a PR — skipping any (event, repo) pair already in
`pull_requests`

.

**This now genuinely runs on a schedule**, not just on-demand: [.github/workflows/scheduled-scan.yml](/rtsdque/sentinel/blob/main/.github/workflows/scheduled-scan.yml)
runs it every 6 hours via GitHub Actions cron (plus a manual `workflow_dispatch`

trigger for
testing), reading secrets from the Actions secrets store rather than a local `.env`

. Secret names
can't start with `GITHUB_`

(a hard GitHub restriction), so they're stored as `SENTINEL_*`

and mapped
to the expected env var names inside the workflow's `env:`

block. The private key is provided as
raw content via `GITHUB_APP_PRIVATE_KEY`

in CI (no local file to point a path at); local dev keeps
using `GITHUB_APP_PRIVATE_KEY_PATH`

— `loadPrivateKeyContent()`

in
[src/github/app.ts](/rtsdque/sentinel/blob/main/src/github/app.ts) checks the inline content first, falls back to the path.

**Validated live in both places, not just typechecked:**

- Locally: backfilled the real
`basil`

track (33 entries via deterministic parsing, zero LLM cost), removed one known-good entry to simulate it being newly discovered, ran the orchestrator against the real`rtsdque/app-test`

repo — it correctly found exactly the 1 new entry (not the other 32), interpreted only that one, cloned the repo fresh, opened a correct PR, and recorded it. Running it again immediately found 0 new entries and did nothing, confirming the dedup path. - In the cloud: pushed to a real (private) GitHub repo,
[rtsdque/sentinel](https://github.com/rtsdque/sentinel), set the 5 required secrets, and triggered the workflow for real via`workflow_dispatch`

— it connected to Neon from GitHub's own infrastructure, correctly found 0 new entries on`basil`

(already fully processed), and exited clean. Confirms the whole chain — secrets,`npm ci`

, DB connectivity, cloning — works outside this machine, not just here. - Multi-repo: registered a second fixture,
[examples/sample-monorepo-repo](/rtsdque/sentinel/blob/main/examples/sample-monorepo-repo)(pushed to its own repo,`rtsdque/sentinel-test-monorepo`

) — a real npm workspace monorepo with the Stripe client constructed in one package and imported by another via a package alias (`@fixture/stripe-client`

), the exact shape found in Documenso. Until this test, that pattern had only ever been scanned read-only, never run through fix generation and PR creation. Re-ran the same balance-settings change against both watched repos in a single orchestrator invocation — it correctly opened two independent, correctly-scoped PRs (`app-test#5`

,`sentinel-test-monorepo#1`

), each with the right diff, both logged against the right repo id in`pull_requests`

.

**Known gap surfaced by this design, not yet resolved**: a watched repo is only checked against
events discovered *after* it was registered. Adding a new repo to `watched_repos`

doesn't scan it
against events already in `change_events`

— there's no "backfill this repo against the last N
known changes" path yet. Fine for the steady-state case (repos stay registered, watcher runs
continuously), but a gap for onboarding a repo after some changes have already been seen.

**This repo's own visibility**:[rtsdque/sentinel](https://github.com/rtsdque/sentinel)is currently private. Going public is a deliberate later decision, not an accident to guard against — but worth knowing the code,`data/stripe/*.json`

, and migration history are all sitting in a repo that isn't public yet, so nothing here has had public scrutiny.**Scanner/fixer scope**: TypeScript/JavaScript + Stripe only. No Python, no other vendors yet.** AST heuristics, not type-checker provenance**: both the scanner's context guards and the fixer's statement-boundary matching are heuristics that can be fooled by unusual code shapes. Treat every match and diff as a candidate for review, not proof. Cross-file client resolution handles direct exports, re-export chains, and monorepo package-alias imports (confirmed against both Cal.com and Documenso) — but only once the repo's dependencies are actually installed (see Phase 3 above); a bare`git clone`

isn't enough for workspace monorepos. Still untested: computed member access (`stripe["oauth"]["token"]`

), a client factory function instead of a bare exported instance. Bare (path-less) rename targets with a common English-word leaf (`name`

,`filter`

,`id`

) remain a known false-positive risk for property-access matching — surfaced via`lowConfidence: true`

on the`ScanMatch`

/`FixCandidate`

, not eliminated; fully closing this needs type-checker provenance (does the receiver actually originate from a Stripe response?), which is future work.**One PR per file-touching event, sequential**:`open-pr.ts`

doesn't yet dedupe against a PR that already exists for the same branch beyond a clear error message — no update-existing-PR flow yet.