{"slug": "the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone", "title": "The bug every AI coding tool ships, and how to prove it is gone", "summary": "A developer released AuditAI Scanner, an open-source tool that detects a common authorization flaw in AI-generated code where service-role keys bypass row-level security, letting one tenant read another's data. The scanner reproduces each finding in an offline sandbox with synthetic tenants, promoting issues from candidate to confirmed only when a request like Alice reading Bob's invoice returns 200 before a fix and 403 after. It reports a coverage line instead of a score and runs locally via npx auditai-scan.", "body_md": "The scanner is open source and the rules are readable before you trust a single finding: [github.com/audit0/auditai-scanner](https://github.com/audit0/auditai-scanner). This post is about the one bug it was built for.\n\nAsk any AI coding tool for an invoicing app and you will get something close to this route handler.\n\n``` js\n// app/api/invoices/[id]/route.ts\nimport { createClient } from \"@supabase/supabase-js\";const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_SERVICE_ROLE_KEY!);\n\nexport async function GET(_req: Request, { params }: { params: { id: string } }) {\n  const { data } = await supabase.from(\"invoices\").select(\"*\").eq(\"id\", params.id).single();\n  return Response.json(data);\n}\n```\n\nIt works. It passes review if the reviewer is reading for \"does this return an invoice\". The service-role key bypasses row level security, and the query filters on `id` alone, so Alice can read Bob's invoice by changing a number in the URL.\n\nThis is not an exotic bug. It is the default outcome of asking for a feature and not asking who is allowed to see it, and it shows up over and over in apps built with Lovable, Bolt, v0, Cursor and Claude Code.\n\nA scanner that prints \"possible authorization issue\" at this line has told you almost nothing. You still have to open the file, work out whether the route is reachable, whether some middleware already blocks it, and whether the id is scoped somewhere you did not read. Most teams do that work once, find two false positives, and stop reading the tool's output.\n\n\"The model says it fixed it\" is worse. A model that both writes the fix and grades the fix is not evidence of anything.\n\nSo we picked a harder bar. A finding stays `likely` until something outside the model reproduces it.\n\nWe copy the app into a sandbox with no internet access, seed two synthetic tenants, and send the same request twice.\n\n```\nproof · AUDIT-001\nGET /api/invoices/42 as Alice\n✗ before fix: 200 OK\n✓ after fix:  403 Forbidden\n```\n\nAlice is not supposed to see invoice 42. Before the fix the endpoint hands it over. That is the evidence. It is also, conveniently, a test.\n\nThe fix is the smallest one that closes the path, not a refactor:\n\n``` js\n-const supabase = createClient(process.env.SUPABASE_URL!, process.env.SUPABASE_SERVICE_ROLE_KEY!);\n+import { createServerClient } from \"@/lib/supabase/server\"; export async function GET(_req: Request, { params }: { params: { id: string } }) {\n-  const { data } = await supabase.from(\"invoices\").select(\"*\").eq(\"id\", params.id).single();\n+  const supabase = await createServerClient();\n+  const { data, error } = await supabase.from(\"invoices\").select(\"*\").eq(\"id\", params.id).single();\n+  if (error || !data) return new Response(\"Forbidden\", { status: 403 });\n   return Response.json(data);\n }\n```\n\nThe request now runs as the signed-in user, so the table's row level security policy decides the answer instead of the route.\n\nThe regression test is the reproduction, kept:\n\n``` js\nit(\"does not let one tenant read another tenant's invoice\", async () => {\n  const res = await fetch(`/api/invoices/${bobInvoiceId}`, { headers: aliceAuth });\n  expect(res.status).toBe(403);\n});\n```\n\nIt failed before the fix. It passes after. If someone reintroduces the service-role client next quarter, it fails again.\n\nEvery result ends with a coverage line rather than a score:\n\n```\nchecked 12 routes · verified 2 · confirmed 1 · unverified 0\n```\n\n`unverified` is not a failure to hide. It is the set of findings we could not reproduce, and we would rather show you the number than dress it up. Findings move from `candidate` to `likely` to `confirmed` only on evidence, and a fix is `verified` only when the regression test failed before it and passes after it, the existing suite still passes, and a deterministic rescan no longer sees the path.\n\nWe measure precision on repositories we have never seen, and publish the number instead of claiming one.\n\n```\nnpx auditai-scan .\n```\n\nIt runs locally, needs no account, and sends nothing anywhere. Source, rules and the eval fixtures are here: [github.com/audit0/auditai-scanner](https://github.com/audit0/auditai-scanner). The hosted product that reproduces, fixes and proves is at [auditai.sh](https://auditai.sh?utm_source=devto&utm_medium=social&utm_campaign=launch).\n\nIf you have a Next.js + Supabase app and want us to find and prove one real authorization bug in it for free, ten design-partner slots are open.", "url": "https://wpnews.pro/news/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone", "canonical_source": "https://dev.to/auditai/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone-55pf", "published_at": "2026-09-12 17:24:38+00:00", "updated_at": "2026-09-12 17:49:29.235408+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products", "ai-safety"], "entities": ["AuditAI Scanner", "Supabase", "Lovable", "Bolt", "v0", "Cursor", "Claude Code"], "alternates": {"html": "https://wpnews.pro/news/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone", "markdown": "https://wpnews.pro/news/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone.md", "text": "https://wpnews.pro/news/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone.txt", "jsonld": "https://wpnews.pro/news/the-bug-every-ai-coding-tool-ships-and-how-to-prove-it-is-gone.jsonld"}}