A security scan says your AI-built app is safe. Does sign-up actually work?

A review of five security scanners aimed at AI-built apps — PreflightVibe, PreflightX, Prodable, Shippable and SafeWeave, whose pages were read on 20 Oct 2026 — found that none of their pages says it tests sign-up, data saving or phone layout, and the ones that state whether they log in say they do not. The scanners check what a stranger can grab without logging in, such as keys in shipped JavaScript, Supabase/Firebase tables readable with the public key, and exposed .env or .git files, while a separate hands-on test of apps built with Lovable, Bolt, Base44 and v0 on free plans on 16 Oct 2026 found mouse-only edit and delete buttons (Bolt), new users stuck at sign-up and sessions that survived logout (Base44), and 6-character passwords accepted (Lovable). All four apps passed the two-account test, with a second user unable to see or change the first user's data.

A security scan says your AI-built app is safe. Does sign-up actually work? A wave of free scanners for AI-built apps will grade your live URL in under a minute. They're useful. But read their own pages and they're mostly answering one question: what can a stranger grab without logging in? Whether your app actually works for a real user is a different test. Security scan: " What can a stranger take?" It reads what your site sends to any browser and makes a few harmless requests: keys in the JavaScript, database tables open to the public key, downloadable .env or .git files, API routes that answer without login, security headers. Logged out, no writes. Functional test: "Does it work for a user?" It uses the app the way a customer would: sign up, log in, create and edit data, tap through it on a phone, log out. It finds the broken button, the stuck sign-up and the page that crashes on an emoji. It can't see a secret key sitting in your code. A security scan can give an A to an app whose sign-up is broken, and a functional test can pass an app that leaks a key. You need both. On 20 Oct 2026 we read the pages of five scanners aimed at vibe-coded apps: PreflightVibe, PreflightX, Prodable, Shippable and SafeWeave. This compares methods only what each page says , not quality. We haven't run them on a test app, and none of them paid us. They overlap heavily on secrets and open data: keys in shipped code, Supabase/Firebase tables or buckets readable without login, API routes that answer without a session. Good, because those leaks expose everyone at once. None of their pages says it tests sign-up, saving data or the phone layout. The ones that say whether they log in say they don't. PreflightVibe is the most direct: it says it can't test what one user can see of another user's data, and tells you to do a two-account test yourself. Full matrix 13 questions × 6 tools, with prices as listed and links to each source page : Security scan vs functional test. Disclosure: we run one of the tools in that matrix a free functional "Scan my app" beta . It has the same blind spot: it never logs in and blocks every write, and it doesn't look for keys or open tables. On 16 Oct 2026 we built the same multi-user to-do app with Lovable, Bolt, Base44 and v0 on free plans one run each and tested every app by hand with real accounts. The problems we found: Buttons that only exist for a mouse Bolt . Edit and delete appeared only on hover, so on a phone they were invisible. Nothing leaked, so a security check has nothing to find. New users stuck at sign-up Base44 . A new user who closed the email-code page found no way to enter or resend the code from the login screen. You only see this by signing up, closing the tab and coming back. Logging out didn't end the session Base44 . The old access token still worked on the server after logout. A security bug, but a logged-out scan never holds a token to replay. Very short passwords accepted Lovable . 6-character passwords went through. Finding it means actually creating an account. None of these was a leaked key or an open table. And on the check that matters most, all four apps passed: a second user couldn't see or change the first user's data. That's one run of a simple app, not a promise about yours. One outside security scan on your live URL, for secret keys, tables readable with your public key and exposed files. Fix anything critical first. A functional pass over your pages: phone layout, buttons, junk input in forms, console and network errors. The logged-in checks, by hand: sign up as a brand-new user badly , log out and replay the old session, and the two-account test about 10 minutes with two browser windows . Again after every big change. Each new prompt to your AI builder can quietly undo a fix. The step-by-step version, with a receipt for each step where we have one: 12-step pre-launch checklist for AI-built apps. Originally published on VibeTierList, where we test AI app builders on the same task and publish the receipts. Paying never changes a ranking. Key Takeaways - •Logged out, no writes. - •This story was reported by Dev.to , covering developments in the dev space. - •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage. 📖 Continue reading the full article: Read Full Article on Dev.to → https://dev.to/vibetierlist/a-security-scan-says-your-ai-built-app-is-safe-does-sign-up-actually-work-5fe