Building an AI-drafted changelog tool with a GitHub App and Claude, solo

Developer built Chngd, a hosted changelog tool that uses Claude Code to draft customer-facing changelog entries from GitHub commits and PRs. The tool filters internal-only commits before sending them to the model, and the developer chose a GitHub App over OAuth+PAT to support per-repo webhooks for auto-sync. A critical security issue was discovered: Supabase's Row-Level Security was disabled by default, exposing all tables via the public anon key, which was fixed after a monitoring alert.

I've been building a hosted changelog tool called Chngd, mostly solo, using Claude Code as a daily collaborator rather than an autocomplete tool. Here's what the actual build looked like, technically, and a mistake that turned into the most useful lesson so far. Most changelog tools ask you to write the changelog. Chngd tries to remove that step: connect a GitHub repo, and it drafts a customer-facing changelog entry from your recent commits and PRs. You review, edit, and publish. Nothing goes out automatically. The pipeline is simple on paper: chore: , ci: , test: , docs: prefixes, skip changelog The filtering step matters more than I expected. Sending raw commit logs to a model produces a changelog that reads like a commit log with better grammar. Filtering internal-only commits first, before the model ever sees them, is what actually makes the output read like something a customer would want. The lazier option was OAuth plus a personal access token. It's faster to build, maybe half a day faster. I went with a GitHub App instead, and the reason came down to one question: what does the paid tier actually need? The free tier is a manual "sync now" button. The paid tier auto-syncs on push, via webhook, and auto-drafts never auto-publishes, that stays a human action . That's fundamentally a GitHub App feature, not an OAuth+PAT one, because per-repo webhooks and installation-scoped access are how GitHub Apps work. Building it on OAuth+PAT first would have meant asking every existing user to redo their connection later. Paying the extra setup cost upfront avoided a migration I'd have hated doing after launch. This is the part worth writing about honestly instead of skipping. I'm using Supabase for Postgres plus Auth, but all the actual data access happens server-side through Drizzle, not through Supabase's client libraries. Because of that, I treated Row-Level Security as something that applied to a pattern I wasn't using, and didn't think hard about it. That was wrong. Supabase's REST API PostgREST sits in front of every table regardless of how your own server code talks to the database. The public anon key is baked into client-side JS by design, it's meant to be public. Without RLS enabled, that anon key can read and write any table with no policy in place, completely independent of whatever access patterns your app code uses. Every table was exposed like this for weeks before I caught it, via a monitoring alert I'd nearly missed. I audited for tampering none found , enabled RLS on all six tables, verified with both a direct Postgres query and a live curl test against the REST endpoint with the anon key, and set up Slack alerting so a future alert doesn't depend on me checking email. The lesson: RLS is off by default in Supabase, and "I don't use the client library for this" is not the same as "this table is protected." If you're using Supabase as Postgres+Auth with a server-side ORM, check this today. It costs one migration and a few minutes. Live at chngd.dev https://chngd.dev , test-mode-to-live Stripe billing, GitHub App install flow, embeddable widget, email notifications on publish. Built and shipped solo. Happy to answer questions about any of the above, especially the Claude integration or the GitHub App setup if anyone's weighing that same decision.