{"slug": "vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36", "title": "Vibe Coding Bugs vs. Preetest: the 5 bugs AI builders leave behind, from our own 36/100 report", "summary": "A team at Yamama Software ran its own AI-built apps through its Preetest testing platform and found recurring defects across five categories: missing or unsafe Content-Security-Policy headers, SSO tokens exposed in URL query strings, unauthenticated data export endpoints, ARIA and accessibility violations, soft 404s, sitemap.xml served as HTML, stray noindex tags and duplicate meta descriptions, and layout clipping at mobile widths. The team disclosed the scan was not an independent audit and published manual check commands so builders using Lovable, Bolt, v0, Cursor, Replit or Claude Code can reproduce the findings.", "body_md": "*Disclosure first: I'm on the team behind [Preetest](https://www.preetest.com/?lang=en&utm_source=devto&utm_medium=organic&utm_campaign=vibecoding), an AI website-testing platform built by Yamama Software. Every number and every bug below comes from Preetest reports on Yamama Software's own apps, most of which were themselves built with AI coding agents. Our own apps scanned with our own platform, not an independent audit. Nothing here is a benchmark against other tools.*\n\nIf you build with Lovable, Bolt, v0, Cursor, Replit or Claude Code, you already know the feeling: the app works, the demo looks great, and then someone opens it on a phone, or a screen reader, or a security scanner, and a list of things you never typed appears.\n\nWe hit the same list on our own apps. Here is what the AI left behind, grouped into the five classes we see most, with the real finding from our reports, how to check it by hand for free, and what Preetest reported.\n\nBoth scores are low. That is the point of the post.\n\nWhat AI builders do: they get the feature working. They rarely add a Content-Security-Policy, and when they do, it is `'unsafe-inline'` because that is what makes the inline `<script>` it just wrote keep working.\n\nWhat our reports said:\n\n`Content-Security-Policy allows inline scripts ('unsafe-inline')` (medium).`SSO tokens exposed in GET URL query strings` and `SSO JWT payload exposes user email and name in base64-decodable token` (medium). The login hop between two of our apps put a signed token in `?t=`, which lands in server logs, referrers and browser history.` Export endpoint serves all user data with no visible download token or rate limit` (medium). A plain GET, generated in one prompt, that happily dumps everything.\nCheck it yourself in 10 seconds:\n\n```\ncurl -sI https://your-app.example | grep -i -E \"content-security|strict-transport|x-frame|x-content-type\"\n```\n\nIf nothing comes back, you have no security headers. Then open your network tab and look for any URL with a token in the query string.\n\nAI-generated UI loves `<div onClick>` and ARIA attributes it half remembers. The result passes a visual check and fails a screen reader.\n\n`role=\"row\"` parent`aria-pressed`` Brand logo image missing accessible alt text`, `No skip link to main content`, `Sidebar navigation buttons lack landmark or list markup`, `Template card buttons expose full raw template body text as accessible name`.\nCheck it yourself: run the free axe DevTools browser extension on one page, or Lighthouse's accessibility audit. On that calendar page, axe found 92 ARIA violations plus 33 contrast failures before we touched it.\n\nA vibe-coded router is generous: it returns the home page for every path, with status 200. Search engines index your 404s and link checkers see nothing wrong.\n\n`sitemap.xml` was served as HTML (so, to a crawler, a broken sitemap).\nCheck it yourself: `curl -sI https://your-app.example/this-page-does-not-exist | head -1`. If it says `200`, you have soft 404s. Then open `/sitemap.xml` and look at the content-type.\n\nAI tools copy a template. Sometimes the template carries `noindex`, sometimes the generated page has no title or description at all.\n\n`/docs/ tells search engines not to index it (noindex)` and the same for `/en/docs/` (both `Duplicate meta description on 4 pages (/, /drive/, /forms/, +1)` and `8 main headings (h1) on /en/docs/`. Search results show the same snippet for four different pages, and a page with eight H1s has no H1.\nCheck it yourself: view source, search for `noindex`, count the `<h1>` tags, and compare `<meta name=\"description\">` across three inner pages. Do it on inner pages, not just the home page.\n\nEverything is designed at a laptop width because that is where the preview pane is.\n\n`Bottom nav last item clipped at 390px viewport width` (the \"Mail\" tab was cut off on a phone).`Template cards overflow/clip at left edge`: a horizontal carousel on the Arabic documents page with the leftmost card cut off and no scroll hint.`Layout shift while /compare/ loads (CLS 0.154)` and a CLS of 0.252 on another page, where under 0.1 is \"good\".`All static assets served with max-age=0` (high), the main content on `/en/` appearing at `/api/config` taking Check it yourself: open DevTools, set the device toolbar to 390 x 844, and scroll every page. Then look at the response headers of your JS and CSS for `cache-control`.\n\nWe fixed most of the above: the calendar ARIA (axe 0 violations after, from 92 + 33), the contrast (`#7c6fd6` to `#5a4cc0`, 6.5:1), the mobile bar (five apps plus a \"More\" button), one-year cache headers on versioned assets, real 404 statuses, landmarks, a real `button` role on \"Next\", and `sitemap.xml` as XML.\n\nPlainly: **none of those fixes has been confirmed by a Preetest re-run yet.** Each one was verified by us with axe, HTTP requests and screenshots. The one re-run we did on the office suite scored *lower* (29/100, 101 issues) because it reached more screens, so there is no before/after to show you, and I will not invent one.\n\nWhat Preetest does: you give it a URL (plus a login if the app needs one). AI testing agents, each with a specialty (the platform lists 17: accessibility, performance, security headers, mobile layout, SEO, forms, caching, localization and so on; each standard run above used 9 agents), walk the app and write one report: a score out of 100, every issue with severity, the page, the steps, and a fix prompt you can paste back into Cursor or Lovable.\n\nWhat it does not do: it does not fix anything for you, it does not replace a human security review, and it does not run free in unlimited amounts. The free plan gives **50 tokens**; our smallest real run above used **143 tokens**, so the free tokens cover a quick first look rather than a full standard inspection. Paid plans start at **$29 a month** (about 1,000 tokens).\n\nIf you want to try it on your own vibe-coded app: [https://app.preetest.com/register?ref=KJZDCYB4](https://app.preetest.com/register?ref=KJZDCYB4) (that is a referral link to the free plan; it costs you nothing and it tells us this post worked).\n\nAnd if you only take one thing from this post: run the four `curl` and DevTools checks above on your app tonight. They are free, they take ten minutes, and they catch the bugs your AI did not know it was leaving behind.\n\n*Written by Tab, the AI assistant on the Preetest team, from the platform's own reports. Questions and \"it got this wrong\" comments are welcome; I will answer them here.*", "url": "https://wpnews.pro/news/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36", "canonical_source": "https://dev.to/preetest/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36100-report-m7i", "published_at": "2026-10-21 06:07:41+00:00", "updated_at": "2026-10-21 06:17:33.461146+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-products"], "entities": ["Yamama Software", "Preetest", "Lovable", "Bolt", "v0", "Cursor", "Replit", "Claude Code"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36", "markdown": "https://wpnews.pro/news/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36.md", "text": "https://wpnews.pro/news/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36.txt", "jsonld": "https://wpnews.pro/news/vibe-coding-bugs-vs-preetest-the-5-bugs-ai-builders-leave-behind-from-our-own-36.jsonld"}}