AI coding assistants are fast. They're also surprisingly consistent at making the same class of structural mistakes.
After scanning hundreds of AI-generated files, I kept seeing the same patterns:
def save_user(data):
validate(data)
return {"status": "saved"} # no INSERT, no UPDATE, nothing
async def fetch_data(url):
return requests.get(url) # synchronous, blocks the event loop
def analyze_sentiment(text):
return {} # zero logic
These aren't random bugs. They're structural patterns that appear across languages and models — GPT-4, Claude, Gemini, Copilot. The AI writes code that looks correct at a glance but breaks at runtime.
The problem: existing scanners weren't designed for this. Bandit and Semgrep catch security vulnerabilities. They don't check whether your save_user()
actually saves.
AINAScan — a deterministic AST scanner with:
| Pattern | What it catches |
|---|---|
MISSING_WRITE |
|
| save/store function with no DB write | |
FAKE_ASYNC |
|
| async def with no await | |
STUB_SKELETON |
|
function that just returns {} or None |
|
DEAD_CALL_RESULT |
|
| calls 3 services, ignores all return values | |
HARDCODED_TABLE |
|
| 40-key dict replacing what should be a DB query | |
INPUT_OUTPUT_DISCONNECTED |
|
| params never used in function body | |
TRIVIAL_IF_CHAIN |
|
| 7+ elif branches with no DB lookup | |
MOCK_PATTERN |
|
| MagicMock in production code | |
EMPTY_EXCEPT |
|
| except: pass swallowing errors silently | |
MISSING_ERROR_HANDLING |
|
| external API calls with no try/catch | |
TRIVIAL_ASSERT |
|
| assert True in tests | |
TODO_PLACEHOLDER |
|
| TODO/FIXME left in production | |
PARAM_SHADOW |
|
| parameter shadowed by local variable | |
SHORT_PASSTHROUGH |
|
| wrapper that adds no value | |
CONST_SQL_NO_PARAM |
|
| SQL WHERE with hardcoded value |
curl -X POST https://pleasing-transformation-production-90c2.up.railway.app/v1/scan \
-H "X-API-Key: vg_free_test" \
-F "file=@your_agent.py"
Response looks like:
{
"passed": false,
"block_count": 2,
"warn_count": 1,
"issues": [
{
"kind": "MISSING_WRITE",
"severity": "BLOCK",
"line": 12,
"detail": "function 'save_user' claims to save but contains no DB write call"
}
]
}
name: VibeGuard Security Scan
on: [pull_request]
jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: Moonsehwan/aina-vibeguard-action@v1
with:
api-key: ${{ secrets.VIBEGUARD_KEY }}
fail-on-block: 'true'
One thing I spent a lot of time on: upstream sanitizer detection.
Before flagging a path traversal issue, the scanner checks 60 lines before the sink for guard patterns:
if '..' in path:
return 400 # scanner sees this
filepath = open(path) # and downgrades BLOCK to WARN
This cut the false positive rate on well-maintained open source repos (Django, FastAPI, celery) from ~40% down to near zero.
I also tested on 10 repos with 100k+ GitHub stars — 0 false positives on legitimate code.
vg_free_test
Questions welcome — especially curious what vibe-coding patterns others are seeing in their AI-generated codebases.