{"slug": "the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them", "title": "The 15 bugs AI coding assistants generate over and over (and a scanner that catches them)", "summary": "A developer created AINAScan, a deterministic AST scanner that detects 15 structural bug patterns commonly generated by AI coding assistants like GPT-4, Claude, Gemini, and Copilot. The scanner catches issues such as fake async functions, missing database writes, and stub skeletons, and includes upstream sanitizer detection to reduce false positives. Tests on 10 repos with 100k+ GitHub stars yielded zero false positives on legitimate code.", "body_md": "AI coding assistants are fast. They're also surprisingly consistent at making the same class of structural mistakes.\n\nAfter scanning hundreds of AI-generated files, I kept seeing the same patterns:\n\n```\n# Pattern 1: MISSING_WRITE\n# AI generates a save function that never actually saves\ndef save_user(data):\n    validate(data)\n    return {\"status\": \"saved\"}   # no INSERT, no UPDATE, nothing\n\n# Pattern 2: FAKE_ASYNC\n# async keyword with no await anywhere\nasync def fetch_data(url):\n    return requests.get(url)     # synchronous, blocks the event loop\n\n# Pattern 3: STUB_SKELETON\n# Placeholder that looks complete but does nothing\ndef analyze_sentiment(text):\n    return {}                    # zero logic\n```\n\nThese aren't random bugs. They're **structural patterns** that appear across languages and models — GPT-4, Claude, Gemini, Copilot. The AI writes code that *looks* correct at a glance but breaks at runtime.\n\nThe problem: existing scanners weren't designed for this. Bandit and Semgrep catch security vulnerabilities. They don't check whether your `save_user()`\n\nactually saves.\n\n**AINAScan** — a deterministic AST scanner with:\n\n| Pattern | What it catches |\n|---|---|\n`MISSING_WRITE` |\nsave/store function with no DB write |\n`FAKE_ASYNC` |\nasync def with no await |\n`STUB_SKELETON` |\nfunction that just returns `{}` or `None`\n|\n`DEAD_CALL_RESULT` |\ncalls 3 services, ignores all return values |\n`HARDCODED_TABLE` |\n40-key dict replacing what should be a DB query |\n`INPUT_OUTPUT_DISCONNECTED` |\nparams never used in function body |\n`TRIVIAL_IF_CHAIN` |\n7+ elif branches with no DB lookup |\n`MOCK_PATTERN` |\nMagicMock in production code |\n`EMPTY_EXCEPT` |\nexcept: pass swallowing errors silently |\n`MISSING_ERROR_HANDLING` |\nexternal API calls with no try/catch |\n`TRIVIAL_ASSERT` |\nassert True in tests |\n`TODO_PLACEHOLDER` |\nTODO/FIXME left in production |\n`PARAM_SHADOW` |\nparameter shadowed by local variable |\n`SHORT_PASSTHROUGH` |\nwrapper that adds no value |\n`CONST_SQL_NO_PARAM` |\nSQL WHERE with hardcoded value |\n\n```\ncurl -X POST https://pleasing-transformation-production-90c2.up.railway.app/v1/scan \\\n  -H \"X-API-Key: vg_free_test\" \\\n  -F \"file=@your_agent.py\"\n```\n\nResponse looks like:\n\n```\n{\n  \"passed\": false,\n  \"block_count\": 2,\n  \"warn_count\": 1,\n  \"issues\": [\n    {\n      \"kind\": \"MISSING_WRITE\",\n      \"severity\": \"BLOCK\",\n      \"line\": 12,\n      \"detail\": \"function 'save_user' claims to save but contains no DB write call\"\n    }\n  ]\n}\nname: VibeGuard Security Scan\non: [pull_request]\n\njobs:\n  scan:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: Moonsehwan/aina-vibeguard-action@v1\n        with:\n          api-key: ${{ secrets.VIBEGUARD_KEY }}\n          fail-on-block: 'true'\n```\n\nOne thing I spent a lot of time on: upstream sanitizer detection.\n\nBefore flagging a path traversal issue, the scanner checks 60 lines before the sink for guard patterns:\n\n```\nif '..' in path:\n    return 400              # scanner sees this\nfilepath = open(path)       # and downgrades BLOCK to WARN\n```\n\nThis cut the false positive rate on well-maintained open source repos (Django, FastAPI, celery) from ~40% down to near zero.\n\nI also tested on 10 repos with 100k+ GitHub stars — 0 false positives on legitimate code.\n\n`vg_free_test`\n\nQuestions welcome — especially curious what vibe-coding patterns others are seeing in their AI-generated codebases.", "url": "https://wpnews.pro/news/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them", "canonical_source": "https://dev.to/_55c9ae90dd2b13bd715f5/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-catches-them-2h90", "published_at": "2026-06-21 01:29:37+00:00", "updated_at": "2026-06-21 02:36:54.966479+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models", "ai-products"], "entities": ["AINAScan", "GPT-4", "Claude", "Gemini", "Copilot", "Django", "FastAPI", "celery"], "alternates": {"html": "https://wpnews.pro/news/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them", "markdown": "https://wpnews.pro/news/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them.md", "text": "https://wpnews.pro/news/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them.txt", "jsonld": "https://wpnews.pro/news/the-15-bugs-ai-coding-assistants-generate-over-and-over-and-a-scanner-that-them.jsonld"}}