Your AI says it's done. Shipcheck tells you if it's ready to ship.
npx ai-shipcheck .
No signup · No API key · No source-code upload · Runs locally
NOT READY 61/100 across 41 assessed checks my-app
Detected: Next.js, React, Supabase, Vitest, Next.js App Router · TypeScript · tests present
Assessed: 41 of 63 checks run · 7/9 categories scored · 9 files · 3 not assessed
Security ██████████░░░░░░ 60 1 finding
Authentication & Authorization ████████░░░░░░░░ 52 2 findings
Database & Data Safety ░░░░░░░░░░░░░░░░ 0 7 findings
Reliability ████████████████ 98 1 finding
Testing ███████████░░░░░ 71 3 findings
Observability ███████████████░ 92 1 finding
Performance ███████████████░ 92 3 findings
Accessibility ················ not assessed
AI Cost & Abuse Controls ················ n/a
Findings
CRITICAL BLOCKER Service-role key read through a browser-visible environment variable
lib/supabase-admin.ts:5:27
│ process.env.NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY!,
lib/supabase-admin.ts reads the Supabase service-role key from a variable with a public
build-time prefix. The value is inlined into the browser bundle, granting every visitor full,
RLS-bypassing access to the database.
Fix: Use the anon key in the browser and rely on row-level security for access control. Keep
the service-role key in server-only code - a route handler, server action, or edge
function - and rotate it immediately if it has ever been in a client bundle.
auth/supabase-service-role-exposure · confidence high · ai-shipcheck explain auth/…
Verdict
· 4 blocking issues must be fixed before deploying. A blocker forces NOT READY regardless of score.
· Weakest category: Database & Data Safety at 0/100 (7 findings).
Scanned 9 files in 28 ms · 41 checks run, 19 not applicable, 3 not assessed · ai-shipcheck v1.0.1
Static analysis of source code - not a security certification.
Real output from fixtures/vulnerable-supabase, trimmed for length.
AI coding tools are very good at producing code that runs, and much less good at producing code that survives production. They report the work as finished either way.
The gaps are consistent and boring: the Supabase table nobody enabled RLS on,
the route handler that writes without checking who is calling, the
NEXT_PUBLIC_
variable holding a secret, the LLM endpoint with no rate limit and no token cap. Shipcheck looks for exactly those, statically, on your machine.
Nine categories, scored independently. 63 rules, each with documentation, a vulnerable fixture, a secure fixture and tests.
| Category | Examples |
|---|---|
| Security | |
Hardcoded credentials, secrets behind NEXT_PUBLIC_ , eval , shell injection, open redirects, permissive CORS, TLS verification disabled, weak crypto |
|
| Auth | |
| Routes that write without an authorisation check, server actions with no auth, browser-only privilege checks, unverified webhooks, unsigned JWTs, exposed service-role keys | |
| Database | |
Tables without row-level security, USING (true) policies, SQL built by interpolation, deletes with no filter, destructive migrations |
|
| Reliability | |
| Swallowed errors, missing timeouts in request paths, unhandled rejections, retries with no backoff, builds set to ignore type errors | |
| Testing | |
No tests, CI missing test/build/typecheck, .only committed, server code with no test referencing it |
|
| Observability | |
No error monitoring, console -only server logging, no React error boundary, handlers that swallow errors |
|
| Performance | |
| Unbounded queries, synchronous I/O in request handlers, N+1 shapes, heavy client imports | |
| Accessibility | |
Missing alt , click handlers on non-interactive elements, unlabelled form controls, positive tabIndex |
|
| AI cost | |
| LLM endpoints with no auth or rate limit, no token cap, request-controlled model selection, provider keys in the browser |
Full catalogue: ** docs/rules** — or run
ai-shipcheck rules
.Stacks it understands: Next.js (both routers), React, Vite, Express, Fastify, Hono, NestJS, Remix, Astro, SvelteKit, Nuxt, Supabase, Firebase, Prisma, Drizzle, Mongoose, Stripe, OpenAI, Anthropic, Vercel AI SDK, LangChain, tRPC, and the common test runners. Framework-specific rules run only when the framework is detected, monorepos included.
npx ai-shipcheck . # scan the current directory
npx ai-shipcheck . --fail-on critical # exit 1 when a critical finding exists
npx ai-shipcheck . --format sarif # also: json, markdown
npx ai-shipcheck explain <rule-id> # why a rule exists, and how to fix it
Findings alone never fail the command — you opt in with --fail-on
or
--min-score
, so adding this to an existing pipeline is a reversible step.
Exit codes: 0
thresholds met, 1
not met, 2
usage error, 3
internal error.
Requires Node.js 22 or newer. Configuration is optional; a
shipcheck.config.json
can exclude paths, disable rules or set thresholds.
Complete reference: ** docs/cli.md** ·
- uses: sinceaihq/ai-shipcheck@v1
with:
fail-on: critical
min-score: 80
Annotates findings inline on the diff, writes a Markdown report to the job
summary, and produces SARIF for code scanning. Outputs score
, verdict
,
critical-count
and high-count
. It is bundled into a single committed file, so a workflow pinned to a tag runs exactly that code with nothing fetched at run time.
Inputs, outputs and recipes: docs/github-action.md
Nothing leaves your machine. No telemetry, no network calls, no upload.Nothing in the scanned repository is executed— no import, noeval
, no install scripts. Files are read as bytes and analysed lexically.Secrets are masked everywhere they could be printed.Scans are bounded, and a truncated scan says so rather than quietly reporting on a fraction of the project.** One runtime dependency.**
Every finding cites a file, a line, a rule, a severity and a confidence — kept deliberately separate, so a heuristic finding is still described as severe while counting for less against the score. Categories that cannot be assessed are excluded rather than awarded a free 100.
The rules were validated against 20 real public repositories pinned by commit SHA. Triaging what they reported cut findings from 5,710 to 2,819 — a 51% reduction, none of it from suppressing projects by name — and surfaced a lexer bug that had been reporting wrong line numbers in any file containing a multi-line comment. Every fix carries a regression test.
** docs/trust.md** ·
·
docs/THREAT_MODEL.mdWorth knowing before you rely on it:
JavaScript and TypeScript only. Other languages are walked, not analysed.Lexical, not semantic. No cross-file reasoning, no type information, and taint tracking that follows a value one hop. A custom auth wrapper it does not recognise can produce a false positive.Express and Fastify routes are not covered by the auth rules. Those understand Next.js conventions today. This is the largest known gap.It cannot see your infrastructure. A table with RLS enabled in a dashboard but absent from migrations is reported asunassessed, not safe.
A clean report means the checks it knows how to make found nothing — not that the code is correct. This is static analysis, not a security certification.
|
CLIConfigurationGitHub ActionScoringTrust modelLimitationsThreat modelArchitectureAdding a ruleReleasingGovernanceFalse positives are the most serious class of bug here. If a rule fires on correct code, report it — the fix ships with a regression test.
Adding a rule is meant to be a single afternoon: write it, add a vulnerable
fixture and a secure fixture, run npm run docs:rules
, run npm run check
. The build fails if a rule never fires on a vulnerable fixture, and fails if any rule fires on a secure one.
CONTRIBUTING.md · good first issues · SUPPORT.md · Discussions
MIT — see LICENSE.
Built by Since AI · builders@sinceai.fi