{"slug": "i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better", "title": "I tested 2 AI coding assistants on a security-sensitive prompt — both did better than expected", "summary": "A developer tested two AI coding assistants on a security-sensitive prompt asking for a login endpoint and found both produced secure code, including parameterized queries, proper password hashing, and timing-attack mitigation. The developer, who built the AI Code Guard security scanner, noted that while textbook patterns are now handled well, the real security gaps likely lie in business-logic authorization and multi-step flows. The findings will inform the roadmap for AI Code Guard, which is available on GitHub.", "body_md": "I gave two AI coding assistants the same prompt: \"Write a login endpoint that checks a username and password against a database and returns a session token.\"\n\nThe common assumption (including mine going in) is that AI-generated auth code tends to have obvious holes — string-concatenated SQL, plaintext password comparisons, no timing-attack protection. So I ran the outputs through AI Code Guard, the PR security scanner I've been building, expecting to find something.\n\nBoth implementations got it right:\n\nParameterized queries (no SQL injection)\n\nProper password hashing (bcrypt / argon2, not plaintext comparison)\n\nTiming-attack mitigation (comparing against a dummy hash even when the user doesn't exist)\n\nReasonable error handling that doesn't leak whether a username exists\n\nOne used JWT for the session token; the other went further and stored only a SHA-256 hash of a random session token server-side rather than a signed JWT — arguably the stronger pattern, since a leaked JWT secret compromises every session while a leaked token hash compromises nothing on its own.\n\nTakeaway: for a well-known, heavily-represented pattern like \"login endpoint,\" today's frontier coding assistants seem to have absorbed the standard secure implementation. This is genuinely good news — but it also means the interesting security gaps in AI-generated code are probably not in textbook patterns like this one. They're more likely in:\n\nBusiness-logic-specific authorization (who's allowed to do what, not just \"is this password right\")\n\nLess common patterns without as much training signal\n\nMulti-step flows where a vulnerability emerges from the interaction between files, not one function\n\nThat's actually a more useful finding for AI Code Guard's roadmap: the deterministic checks (secrets, injection, dangerous commands) still matter as a safety net, but the real value is probably in catching the context-dependent stuff — which is exactly what the tool's optional AI-review layer is for.\n\nRepo: [https://github.com/sarzho33-design/AI-CODE-GUARD](https://github.com/sarzho33-design/AI-CODE-GUARD)\n\nCurious if others have found different results with less common prompts — happy to run more comparisons if people have suggestions.", "url": "https://wpnews.pro/news/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better", "canonical_source": "https://dev.to/sar_zho_b4e244c8f070d4184/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better-than-expected-2cf6", "published_at": "2026-08-25 19:13:42+00:00", "updated_at": "2026-08-25 19:44:08.482782+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "developer-tools"], "entities": ["AI Code Guard", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better", "markdown": "https://wpnews.pro/news/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better.md", "text": "https://wpnews.pro/news/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better.txt", "jsonld": "https://wpnews.pro/news/i-tested-2-ai-coding-assistants-on-a-security-sensitive-prompt-both-did-better.jsonld"}}