{"slug": "show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship", "title": "Show HN: AI Shipcheck – know if your AI-built app is ready to ship", "summary": "AI Shipcheck, a new open-source static analysis tool released as v1.0.1, scans AI-generated code locally to flag production-readiness issues, scoring apps across nine categories and blocking deployment on critical findings. In a demo on a vulnerable Supabase app, it returned a NOT READY verdict of 61/100 across 41 checks, identifying a critical service-role key exposure and 7 database safety findings. The tool runs via `npx ai-shipcheck .` with no signup, API key, or source upload, and includes 63 rules with documentation and fixtures.", "body_md": "**Your AI says it's done. Shipcheck tells you if it's ready to ship.**\n\n```\nnpx ai-shipcheck .\n```\n\n**No signup · No API key · No source-code upload · Runs locally**\n\n```\n   NOT READY   61/100 across 41 assessed checks                        my-app\n\n  Detected: Next.js, React, Supabase, Vitest, Next.js App Router · TypeScript · tests present\n  Assessed: 41 of 63 checks run · 7/9 categories scored · 9 files · 3 not assessed\n\n  Security                        ██████████░░░░░░   60  1 finding\n  Authentication & Authorization  ████████░░░░░░░░   52  2 findings\n  Database & Data Safety          ░░░░░░░░░░░░░░░░    0  7 findings\n  Reliability                     ████████████████   98  1 finding\n  Testing                         ███████████░░░░░   71  3 findings\n  Observability                   ███████████████░   92  1 finding\n  Performance                     ███████████████░   92  3 findings\n  Accessibility                   ················  not assessed\n  AI Cost & Abuse Controls        ················  n/a\n\n  Findings\n\n  CRITICAL BLOCKER Service-role key read through a browser-visible environment variable\n      lib/supabase-admin.ts:5:27\n      │ process.env.NEXT_PUBLIC_SUPABASE_SERVICE_ROLE_KEY!,\n      lib/supabase-admin.ts reads the Supabase service-role key from a variable with a public\n      build-time prefix. The value is inlined into the browser bundle, granting every visitor full,\n      RLS-bypassing access to the database.\n      Fix: Use the anon key in the browser and rely on row-level security for access control. Keep\n           the service-role key in server-only code - a route handler, server action, or edge\n           function - and rotate it immediately if it has ever been in a client bundle.\n      auth/supabase-service-role-exposure · confidence high · ai-shipcheck explain auth/…\n\n  Verdict\n\n  · 4 blocking issues must be fixed before deploying. A blocker forces NOT READY regardless of score.\n  · Weakest category: Database & Data Safety at 0/100 (7 findings).\n\n  Scanned 9 files in 28 ms · 41 checks run, 19 not applicable, 3 not assessed · ai-shipcheck v1.0.1\n  Static analysis of source code - not a security certification.\n```\n\nReal output from fixtures/vulnerable-supabase, trimmed for length.\n\nAI coding tools are very good at producing code that runs, and much less good at producing code that survives production. They report the work as finished either way.\n\nThe gaps are consistent and boring: the Supabase table nobody enabled RLS on,\nthe route handler that writes without checking who is calling, the\n`NEXT_PUBLIC_`\n\nvariable holding a secret, the LLM endpoint with no rate limit\nand no token cap. Shipcheck looks for exactly those, statically, on your\nmachine.\n\nNine categories, scored independently. **63 rules**, each with documentation, a\nvulnerable fixture, a secure fixture and tests.\n\n| Category | Examples |\n|---|---|\nSecurity |\nHardcoded credentials, secrets behind `NEXT_PUBLIC_` , `eval` , shell injection, open redirects, permissive CORS, TLS verification disabled, weak crypto |\nAuth |\nRoutes that write without an authorisation check, server actions with no auth, browser-only privilege checks, unverified webhooks, unsigned JWTs, exposed service-role keys |\nDatabase |\nTables without row-level security, `USING (true)` policies, SQL built by interpolation, deletes with no filter, destructive migrations |\nReliability |\nSwallowed errors, missing timeouts in request paths, unhandled rejections, retries with no backoff, builds set to ignore type errors |\nTesting |\nNo tests, CI missing test/build/typecheck, `.only` committed, server code with no test referencing it |\nObservability |\nNo error monitoring, `console` -only server logging, no React error boundary, handlers that swallow errors |\nPerformance |\nUnbounded queries, synchronous I/O in request handlers, N+1 shapes, heavy client imports |\nAccessibility |\nMissing `alt` , click handlers on non-interactive elements, unlabelled form controls, positive `tabIndex` |\nAI cost |\nLLM endpoints with no auth or rate limit, no token cap, request-controlled model selection, provider keys in the browser |\n\nFull catalogue: ** docs/rules** — or run\n\n`ai-shipcheck rules`\n\n.**Stacks it understands:** Next.js (both routers), React, Vite, Express,\nFastify, Hono, NestJS, Remix, Astro, SvelteKit, Nuxt, Supabase, Firebase,\nPrisma, Drizzle, Mongoose, Stripe, OpenAI, Anthropic, Vercel AI SDK, LangChain,\ntRPC, and the common test runners. Framework-specific rules run only when the\nframework is detected, monorepos included.\n\n```\nnpx ai-shipcheck .                       # scan the current directory\nnpx ai-shipcheck . --fail-on critical    # exit 1 when a critical finding exists\nnpx ai-shipcheck . --format sarif        # also: json, markdown\nnpx ai-shipcheck explain <rule-id>       # why a rule exists, and how to fix it\n```\n\nFindings alone never fail the command — you opt in with `--fail-on`\n\nor\n`--min-score`\n\n, so adding this to an existing pipeline is a reversible step.\nExit codes: `0`\n\nthresholds met, `1`\n\nnot met, `2`\n\nusage error, `3`\n\ninternal\nerror.\n\nRequires Node.js 22 or newer. Configuration is optional; a\n`shipcheck.config.json`\n\ncan exclude paths, disable rules or set thresholds.\n\nComplete reference: ** docs/cli.md** ·\n\n[docs/configuration.md](/sinceaihq/ai-shipcheck/blob/main/docs/configuration.md)\n\n```\n- uses: sinceaihq/ai-shipcheck@v1\n  with:\n    fail-on: critical\n    min-score: 80\n```\n\nAnnotates findings inline on the diff, writes a Markdown report to the job\nsummary, and produces SARIF for code scanning. Outputs `score`\n\n, `verdict`\n\n,\n`critical-count`\n\nand `high-count`\n\n. It is bundled into a single committed file,\nso a workflow pinned to a tag runs exactly that code with nothing fetched at\nrun time.\n\nInputs, outputs and recipes: [docs/github-action.md](/sinceaihq/ai-shipcheck/blob/main/docs/github-action.md)\n\n**Nothing leaves your machine.** No telemetry, no network calls, no upload.**Nothing in the scanned repository is executed**— no import, no`eval`\n\n, no install scripts. Files are read as bytes and analysed lexically.**Secrets are masked** everywhere they could be printed.**Scans are bounded**, and a truncated scan says so rather than quietly reporting on a fraction of the project.** One runtime dependency.**\n\nEvery finding cites a file, a line, a rule, a severity **and a confidence** —\nkept deliberately separate, so a heuristic finding is still described as severe\nwhile counting for less against the score. Categories that cannot be assessed\nare excluded rather than awarded a free 100.\n\nThe rules were validated against **20 real public repositories** pinned by\ncommit SHA. Triaging what they reported cut findings from 5,710 to 2,819 — a\n51% reduction, none of it from suppressing projects by name — and surfaced a\nlexer bug that had been reporting wrong line numbers in any file containing a\nmulti-line comment. Every fix carries a regression test.\n\n** docs/trust.md** ·\n\n**·**\n\n[corpus/TRIAGE.md](/sinceaihq/ai-shipcheck/blob/main/corpus/TRIAGE.md)\n\n[docs/THREAT_MODEL.md](/sinceaihq/ai-shipcheck/blob/main/docs/THREAT_MODEL.md)Worth knowing before you rely on it:\n\n**JavaScript and TypeScript only.** Other languages are walked, not analysed.**Lexical, not semantic.** No cross-file reasoning, no type information, and taint tracking that follows a value one hop. A custom auth wrapper it does not recognise can produce a false positive.**Express and Fastify routes are not covered by the auth rules.** Those understand Next.js conventions today. This is the largest known gap.**It cannot see your infrastructure.** A table with RLS enabled in a dashboard but absent from migrations is reported as*unassessed*, not safe.\n\nA clean report means the checks it knows how to make found nothing — **not**\nthat the code is correct. This is static analysis, not a security\ncertification.\n\n|\n\n[CLI](/sinceaihq/ai-shipcheck/blob/main/docs/cli.md)[Configuration](/sinceaihq/ai-shipcheck/blob/main/docs/configuration.md)[GitHub Action](/sinceaihq/ai-shipcheck/blob/main/docs/github-action.md)[Scoring](/sinceaihq/ai-shipcheck/blob/main/docs/SCORING.md)[Trust model](/sinceaihq/ai-shipcheck/blob/main/docs/trust.md)[Limitations](/sinceaihq/ai-shipcheck/blob/main/docs/LIMITATIONS.md)[Threat model](/sinceaihq/ai-shipcheck/blob/main/docs/THREAT_MODEL.md)[Architecture](/sinceaihq/ai-shipcheck/blob/main/docs/ARCHITECTURE.md)[Adding a rule](/sinceaihq/ai-shipcheck/blob/main/docs/adding-a-rule.md)[Releasing](/sinceaihq/ai-shipcheck/blob/main/docs/RELEASING.md)[Governance](/sinceaihq/ai-shipcheck/blob/main/GOVERNANCE.md)False positives are the most serious class of bug here. If a rule fires on\ncorrect code, [report it](https://github.com/sinceaihq/ai-shipcheck/issues/new?template=false_positive.yml)\n— the fix ships with a regression test.\n\nAdding a rule is meant to be a single afternoon: write it, add a vulnerable\nfixture and a secure fixture, run `npm run docs:rules`\n\n, run `npm run check`\n\n.\nThe build fails if a rule never fires on a vulnerable fixture, and fails if any\nrule fires on a secure one.\n\n[CONTRIBUTING.md](/sinceaihq/ai-shipcheck/blob/main/CONTRIBUTING.md) · [good first issues](https://github.com/sinceaihq/ai-shipcheck/issues?q=is%3Aopen+label%3A%22good+first+issue%22) · [SUPPORT.md](/sinceaihq/ai-shipcheck/blob/main/SUPPORT.md) · [Discussions](https://github.com/sinceaihq/ai-shipcheck/discussions)\n\nMIT — see [LICENSE](/sinceaihq/ai-shipcheck/blob/main/LICENSE).\n\nBuilt by [Since AI](https://sinceai.ai) · [builders@sinceai.fi](mailto:builders@sinceai.fi)", "url": "https://wpnews.pro/news/show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship", "canonical_source": "https://github.com/sinceaihq/ai-shipcheck", "published_at": "2026-08-28 18:19:13+00:00", "updated_at": "2026-08-28 18:48:27.808297+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety", "developer-tools"], "entities": ["AI Shipcheck", "Supabase", "Next.js", "React", "Vitest"], "alternates": {"html": "https://wpnews.pro/news/show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship", "markdown": "https://wpnews.pro/news/show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship.md", "text": "https://wpnews.pro/news/show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship.txt", "jsonld": "https://wpnews.pro/news/show-hn-ai-shipcheck-know-if-your-ai-built-app-is-ready-to-ship.jsonld"}}