Watches vendor API changelogs, detects breaking changes before they hit customer codebases, scans a customer repo for affected usages, generates reviewable fix diffs, and opens a PR — before the sunset date, not after something breaks in production.
See it detect a real breaking change with no account, API key, or database — just Node:
git clone https://github.com/rtsdque/sentinel.git
cd sentinel
npm install
npx tsx src/cli.ts stripe --track acacia
That's it — this scrapes Stripe's real changelog and prints a fully structured breaking-change
event to your terminal, deterministically (no LLM call). Everything past this point (interpreting
changes with an LLM, scanning a codebase, opening PRs, running on a schedule) layers on top and
needs progressively more setup — jump to the phase you want below, or npm test
to run the suite (also needs nothing but Node).
This is a real diff Sentinel generated and opened as a PR, unedited, against a test repo — Stripe nested three Balance Settings request fields under a new required payments object, and Sentinel found the affected call site and proposed the migration on its own:
export async function enableNegativeBalanceDebits() {
return stripe.balanceSettings.update({
- debit_negative_balances: true,
- payouts: {
- schedule: { interval: "daily" },
+ payments: {
+ debit_negative_balances: true,
+ payouts: {
+ schedule: { interval: "daily" },
+ },
},
});
}
The PR body links the upstream changelog entry, states the deadline if there is one, and separately lists anything it couldn't confidently fix — see Phase 4 and Phase 5 below for what makes it refuse rather than guess.
Reasonable question — here's what's actually different, based on a real look at the landscape, not a guess:
| Tool | What it does | Where it stops |
|---|---|---|
| Dependabot (incl. AI-agent remediation) | ||
| Reacts to a known CVE or a new package version in your manifest | Anchored to your dependency file — never reads a vendor's changelog prose, only version diffs | |
| Renovate | ||
Same, plus customDatasources for arbitrary version feeds |
||
| Still just version tracking — no interpretation of what changed or why it breaks you | ||
| oasdiff / openapi-diff | ||
| Diffs two OpenAPI specs, flags breaking changes precisely | Needs the vendor's own spec-versioned CI — doesn't touch your codebase at all | |
| Visualping / Theneo / PageCrawl | ||
| Watch a changelog page, alert on any change | Stops at a notification. You still read it, find the usages, and fix them yourself | |
| ChangeSpec | ||
| An open JSON format for vendors to publish structured change events | A spec waiting for vendor adoption — Stripe doesn't publish in it today, so someone still has to read the prose |
Every one of these is genuinely good at one piece. None of them do the thing this YC RFS actually asked for: watch a vendor's unstructured changelog prose, turn it into a structured breaking change, find every place your specific codebase uses the affected surface, and propose the fix — before your build breaks, not after. That's the gap Sentinel targets, and as far as we could find during a real competitive pass (not a hunch), nobody had shipped the full pipeline yet.
Phases 1–6 built, scoped to Stripe + TypeScript/JavaScript customer repos only (see project scope notes — deliberately not broad).
| Phase | What it does | Entry point |
|---|---|---|
| 1. Change Watcher | Scrapes Stripe's changelog, deterministically parses breaking-change entries into structured ChangeEvent s |
|
src/cli.ts |
||
| 2. Change Interpreter | LLM step (claude-opus-4-8 ) that decomposes prose + the parsed param table into atomic, machine-actionable changes |
|
src/cli.ts --interpret |
||
| 3. Codebase Scanner | AST-based (ts-morph ) scan of a TS/JS repo for usages of the affected API surface, scoped to real Stripe client calls |
|
src/scan.ts |
||
| 4. Fix Generator | Per-match LLM call that proposes a diff or explicitly declines with a reason — never guesses | src/fix.ts |
| 5. PR Bot | GitHub App that applies approved fixes, formats with Prettier, and opens a PR — every PR states what wasn't auto-fixed | src/open-pr.ts |
| 6. Persistence & Scheduling | Postgres-backed "have I seen this" store + a repo registry, so a run only acts on genuinely new changes | src/run-scheduled.ts |
Nothing in this pipeline auto-merges. Every fix is generated for human/PR review; the PR body explicitly separates "proposed changes" from "needs manual review," because there's no customer test suite to verify correctness against.
src/
lib/ fetch wrapper, markdown-table parsing
watcher/stripe/ Phase 1: changelog fetch + parse + ChangeEvent builder
interpreter/stripe.ts Phase 2: LLM decomposition into atomic changes
scanner/typescript.ts Phase 3: ts-morph AST scan for affected usages
fixer/stripe.ts Phase 4: per-match fix generation + apply/consolidate/format
github/
app.ts GitHub App installation auth
pr.ts branch + commit + PR creation
cli.ts / scan.ts / fix.ts / open-pr.ts entry points for each phase
types.ts
data/stripe/ structured ChangeEvent JSON, one file per detected entry
examples/sample-customer-repo/ synthetic fixture repo used to validate the scanner/fixer
npm install
cp .env.example .env
Fill in .env
directly in your editor — never paste secrets into chat. ANTHROPIC_API_KEY
is
needed for Phases 2 and 4; the GITHUB_APP_*
vars are needed for Phase 5 (see below).
Note onon Windows, npm'snpm run <script> -- --flags
:.cmd
shim can silently drop--flag
-style arguments passed after--
. Every command below usesnpx tsx
directly instead, which doesn't have this problem — pass--env-file=.env
if the command needs your API key.
npx tsx src/cli.ts stripe --track acacia
Flags: --track <name>
(default acacia
; current live track is dahlia
), --all
(include
non-breaking entries), --limit N
. Output: one JSON file per ChangeEvent
under data/stripe/
.
Stripe serves clean raw markdown for any doc page via a .md
suffix — no HTML scraping. The index
page flags each entry Breaking
/Non-breaking
; detail pages include a per-language parameter
table (Added
/Removed
) that Phase 1 parses deterministically.
npx tsx --env-file=.env src/cli.ts stripe --track acacia --interpret
Attaches an interpreted
field to each ChangeEvent
: an array of atomic changes
(param_renamed
, param_removed
, method_removed
, response_shape_changed
, etc., each with
from
/to
/description
), a migration summary, and a confidence level. Deduplicates to one canonical entry per target — validated against 10 real breaking changes across 3 release tracks. Stripe's own tables are sometimes incomplete (e.g. list only the new values with no explicit mapping from old ones); the interpreter recovers the full mapping from prose in those cases.
npx tsx src/scan.ts <path-to-customer-repo> --events <eventId1>,<eventId2>
Omit --events
to scan against every interpreted event in data/stripe/
. TypeScript/JavaScript
only. Uses ts-morph
to parse the real AST rather than grep:
Call-argument matches(request params) are scoped by tracing the call's callee chain back to a known Stripe client variable — either constructed locally (new Stripe(...)
) or imported from another local module that constructs one (import stripe from "./server"
, resolved recursively through re-exports viats-morph
'sgetExportedDeclarations()
). Not just "any object literal in a file that happens to import stripe."Both call-argument and property-access matches require the target's parent path segment (when it has one — e.g.billing_mode
forbilling_mode.type
,errors
forAccount.requirements.errors[].code
) to appear in the surrounding context — the receiver expression's text for property access, the enclosing call's callee text for call arguments — guarding generic leaf names liketype
,status
, orcode
from matching unrelated objects.
Both are AST/text heuristics, not full type-checker provenance — meaningfully better than grep, but not infallible. Validated in two stages:
- Against
examples/sample-customer-repo, a fixture with deliberate false-positive traps (an unrelated local field sharing a param name, an unrelated Stripe
Event.type
read) — zero false positives. - Against a real, unmodified production codebase —
Cal.com, a large TypeScript monorepo — which immediately surfaced a real gap the fixture couldn't: Cal.com constructs its Stripe client once in a shared module and imports it everywhere else, a very common production pattern our same-file-only detection completely missed (a falsenegative, confirmed empirically with a synthetic target before fixing). Fixing that then widened the surface for a second real false positive — a generic leaf name (code
) matching an unrelated OAuth call — which is what the context-guard extension above addresses. Re-verified clean on both the fixture and Cal.com after both fixes. - Against a second real codebase —
Documenso, an npm workspace monorepo — which surfaced two more real findings:- The shared Stripe client is imported via a
monorepo package alias(@documenso/lib/server-only/stripe
), not a relative path.ts-morph
's module resolution needs the workspace'snode_modules
symlinks on disk to follow this — a baregit clone
isn't enough;dependencies must be installed (, or every cross-package import silently resolves to nothing.npm install
, ideally with--ignore-scripts
for read-only static analysis of a repo you don't otherwise trust) before scanning a workspace monorepo - Once installed, scanning surfaced a case the context guard can't help with: a
bare rename target with no path segments at all (e.g.name
, or in a live run,filter
) has no context to check, so property-access matching falls back to "anywhere in a file that touches a Stripe client" — which matched an unrelated Prisma call's.name
field in testing, and in a real run against the 10 validated events, matchedArray.prototype.filter()
calls (.data.filter(...)
) against the Billing Alertsfilter
field purely on name collision. Rather than chase a fragile heuristic fix (properly solving this needs type-checker provenance — does the receiver actually originate from a Stripe response? — which is a bigger lift than this pass), matches like this now carrylowConfidence: true
and print with an explicit warning inscan.ts
/fix.ts
output, so they're surfaced for extra scrutiny instead of looking identical to a well-scoped match.
- The shared Stripe client is imported via a
npx tsx --env-file=.env src/fix.ts <path-to-customer-repo> --events <eventId1>,<eventId2>
For each scan match, sends the enclosing statement + the atomic change details to
claude-opus-4-8
and gets back a structured verdict: canGenerateFix
, confidence
, a proposed
replacement (or null
), an explanation, and caveats. Prints two sections: fixes with a proposed diff, and matches flagged for manual review with no diff proposed.
The generator is instructed to refuse rather than guess when a change alters a value's shape, not just its key name (e.g. a boolean becoming a string enum) — validated against exactly that trap and two others; it correctly declined all three with specific, useful reasoning, and correctly generated an accurate diff for a case that was genuinely safe (a pure structural rename with no value-type change).
npx tsx --env-file=.env src/open-pr.ts <path-to-local-clone> --owner <org> --repo <repo> --base main --events <eventId>
Runs scan → fix → apply (with statement-level deduplication when multiple atomic changes land in
the same statement, and a Prettier pass to clean up indentation from spliced-in replacement text)
→ opens one PR per ChangeEvent
. The PR body always includes both the applied changes and the items flagged for manual review — nothing is silently dropped — plus a disclaimer that nothing has been tested.
This touches customer code, so the app is scoped as narrowly as possible:
| Permission | Level | Why |
|---|---|---|
| Contents | Read & write | Create a branch, commit the fix |
| Pull requests | Read & write | Open the PR |
| Metadata | Read (mandatory default) | Required by every GitHub App |
No Administration
, no Actions
, no org-wide access. No webhook subscription needed — this runs on-demand from the CLI, not as an event listener, so you can leave "Webhook" unchecked when registering the app.
- Go to
github.com/settings/apps/new(or your org's
/settings/apps/new
for an org-owned app). - Set the permissions above under "Repository permissions."
- Uncheck "Active" under Webhook (not needed for this phase).
- Under "Where can this GitHub App be installed?", choose "Only on this account" unless you specifically want it public.
- Create the app, note the
App ID shown on its settings page. - Scroll to "Private keys" →
Generate a private key— downloads a
.pem
file. Save it somewhere local (not inside this repo), and setGITHUB_APP_PRIVATE_KEY_PATH
in.env
to that file's path. Never paste the key's contents into chat or commit it anywhere. - Click
Install App, choose a repo you own to test against (ideally a throwaway/test repo, not anything real, for the first run), and select "Only select repositories." - After installing, the URL bar shows
.../installations/<number>
— that number isGITHUB_APP_INSTALLATION_ID
. - Set
GITHUB_APP_ID
andGITHUB_APP_INSTALLATION_ID
in.env
.
I can't do any of this myself — it requires your GitHub account and creates a real, persistent app
registration. Once it's set up, open-pr.ts
only ever acts within that installation's granted repos.
Postgres, via Drizzle ORM. Two ways to get one:
Cloud (needed for real scheduling): a freeNeonor Supabase project — required if you actually want.github/workflows/scheduled-scan.yml
to run unattended, since a GitHub Actions runner needs a database it can reach over the network, not one on your laptop.Local, for just trying it out:docker compose up -d
starts Postgres onlocalhost:55432
(deliberately not the default 5432 — many machines already have a local Postgres service bound to that port, and connections would silently land on the wrong server with a confusing auth error instead of an obvious "port in use"). Then setDATABASE_URL=postgresql://sentinel:sentinel@localhost:55432/sentinel
in.env
. Fine for runningrun-scheduled.ts
by hand; won't work for the cloud cron workflow.
Three tables (src/db/schema.ts):
change_events
— every detected entry, keyed by the same stable id the watcher already computes. Row existence is what "have I seen this before" means.watched_repos
— registry of which repos to scan for which vendor, replacing manually passing--owner
/--repo
on the CLI.pull_requests
— log of PRs Sentinel has opened; prevents re-opening a PR for a (change event, repo) pair already handled, and is the source of truth for the "users" metric (distinct repos with at least one row here).
npm run db:generate # generate a migration from the schema
npm run db:migrate # apply it
npx tsx src/backfill.ts <track> # mark existing history as known WITHOUT interpreting or acting
node --env-file=.env --import tsx src/manage-repos.ts add --owner <o> --repo <r>
node --env-file=.env --import tsx src/manage-repos.ts list
node --env-file=.env --import tsx src/run-scheduled.ts --track <track>
run-scheduled.ts
is the actual orchestrator: check a track for entries not yet in
change_events
→ interpret only the new ones (the cost-saving point of persistence — a backfilled
track costs nothing to re-check) → for each active watched repo, clone it fresh via the GitHub
App's own installation token (never a persisted local path — a scheduled run has no guarantee of
running on the same machine twice, and a cloud runner has no access to anyone's local filesystem at
all), npm install --ignore-scripts
(needed for monorepo package-alias imports to resolve — see
Phase 3), scan, fix, apply, and open a PR — skipping any (event, repo) pair already in
pull_requests
.
This now genuinely runs on a schedule, not just on-demand: .github/workflows/scheduled-scan.yml
runs it every 6 hours via GitHub Actions cron (plus a manual workflow_dispatch
trigger for
testing), reading secrets from the Actions secrets store rather than a local .env
. Secret names
can't start with GITHUB_
(a hard GitHub restriction), so they're stored as SENTINEL_*
and mapped
to the expected env var names inside the workflow's env:
block. The private key is provided as
raw content via GITHUB_APP_PRIVATE_KEY
in CI (no local file to point a path at); local dev keeps
using GITHUB_APP_PRIVATE_KEY_PATH
— loadPrivateKeyContent()
in src/github/app.ts checks the inline content first, falls back to the path.
Validated live in both places, not just typechecked:
- Locally: backfilled the real
basil
track (33 entries via deterministic parsing, zero LLM cost), removed one known-good entry to simulate it being newly discovered, ran the orchestrator against the realrtsdque/app-test
repo — it correctly found exactly the 1 new entry (not the other 32), interpreted only that one, cloned the repo fresh, opened a correct PR, and recorded it. Running it again immediately found 0 new entries and did nothing, confirming the dedup path. - In the cloud: pushed to a real (private) GitHub repo,
rtsdque/sentinel, set the 5 required secrets, and triggered the workflow for real viaworkflow_dispatch
— it connected to Neon from GitHub's own infrastructure, correctly found 0 new entries onbasil
(already fully processed), and exited clean. Confirms the whole chain — secrets,npm ci
, DB connectivity, cloning — works outside this machine, not just here. - Multi-repo: registered a second fixture,
examples/sample-monorepo-repo(pushed to its own repo,rtsdque/sentinel-test-monorepo
) — a real npm workspace monorepo with the Stripe client constructed in one package and imported by another via a package alias (@fixture/stripe-client
), the exact shape found in Documenso. Until this test, that pattern had only ever been scanned read-only, never run through fix generation and PR creation. Re-ran the same balance-settings change against both watched repos in a single orchestrator invocation — it correctly opened two independent, correctly-scoped PRs (app-test#5
,sentinel-test-monorepo#1
), each with the right diff, both logged against the right repo id inpull_requests
.
Known gap surfaced by this design, not yet resolved: a watched repo is only checked against
events discovered after it was registered. Adding a new repo to watched_repos
doesn't scan it
against events already in change_events
— there's no "backfill this repo against the last N known changes" path yet. Fine for the steady-state case (repos stay registered, watcher runs continuously), but a gap for onboarding a repo after some changes have already been seen.
This repo's own visibility:rtsdque/sentinelis currently private. Going public is a deliberate later decision, not an accident to guard against — but worth knowing the code,data/stripe/*.json
, and migration history are all sitting in a repo that isn't public yet, so nothing here has had public scrutiny.Scanner/fixer scope: TypeScript/JavaScript + Stripe only. No Python, no other vendors yet.** AST heuristics, not type-checker provenance**: both the scanner's context guards and the fixer's statement-boundary matching are heuristics that can be fooled by unusual code shapes. Treat every match and diff as a candidate for review, not proof. Cross-file client resolution handles direct exports, re-export chains, and monorepo package-alias imports (confirmed against both Cal.com and Documenso) — but only once the repo's dependencies are actually installed (see Phase 3 above); a baregit clone
isn't enough for workspace monorepos. Still untested: computed member access (stripe["oauth"]["token"]
), a client factory function instead of a bare exported instance. Bare (path-less) rename targets with a common English-word leaf (name
,filter
,id
) remain a known false-positive risk for property-access matching — surfaced vialowConfidence: true
on theScanMatch
/FixCandidate
, not eliminated; fully closing this needs type-checker provenance (does the receiver actually originate from a Stripe response?), which is future work.One PR per file-touching event, sequential:open-pr.ts
doesn't yet dedupe against a PR that already exists for the same branch beyond a clear error message — no update-existing-PR flow yet.