{"slug": "why-relying-on-llms-for-code-can-be-a-security-nightmare", "title": "Why Relying on LLMs for Code Can Be a Security Nightmare", "summary": "Relying on large language models (LLMs) to generate code without human security review can introduce serious vulnerabilities, as LLMs often replicate insecure patterns found in online examples. The article illustrates this with a real-world case where a JavaScript file, likely generated by an LLM, exposed a full email API endpoint and parameters in client-side code, allowing anyone to send arbitrary requests to the backend. The author warns that while security issues in example code are not new, LLMs scale the problem by blindly reproducing these flaws, making human oversight essential for threat modeling and abuse prevention.", "body_md": "Why Relying on LLMs for Code Can Be a Security Nightmare\nTable of Contents\nI wanted to quickly highlight something I have noticed in multiple cases but this time let me explain with a real example.\nWhen code is written purely using LLMs (vibe coded) security often takes a back seat.\nLLMs work on data and most of what is available at training time (essentially scraping the internet) is NOT “secure by default” A lot of examples floating around online are meant to demonstrate functionality not teach security best practices. When those patterns are blindly reused or auto generated by an LLM the same insecure logic ends up in production code.\nThis is NOT my first time stumbling across such issues. Some time ago, I found a bug in Coinbase’s x402 pay-per-view example code (not in the core library itself, just the example). When I reported it through HackerOne, it was marked as informational—since the problem was only in the sample code. That experience stuck with me because it shows that even official documentation can contain insecure examples. If developers copy paste blindly, those vulnerabilities end up in live apps even when the code is not generated using LLM.\nAnd that’s the point: security issues in example code are not new. But with LLMs the problem scales faster, because LLMs often mimic those same insecure patterns.\nNow, let me share a live example I came across recently.\nThe Vulnerable App (details redacted)⌗\nWhile browsing, I came across a JavaScript file on a site hosted on railway.com popular PaaS. The file contained this (lightly paraphrased) code:\n// Form Submission Handling\nconst smtp_api = \"https://<redacted-domain>/send-email\";\nconst phoneNumberPattern = /^(\\\\+\\\\d{1,2}\\\\s?)?(\\\\(?\\\\d{3}\\\\)?[\\\\s.-]?)?\\\\d{3}[\\\\s.-]?\\\\d{4}$/;\nfunction validateForm(name, email, number) {\nif (!name || !email || !number) {\nalert(\"Please fill in all required fields.\");\nreturn false;\n}\nreturn true;\n}\nasync function submitForm(name, email, number, countryCode = '') {\nconst data = {\nname,\nemail,\nnumber,\ncountry_code: countryCode,\ncompany_email: \"<redacted@redacted>\",\nproject_name: \"<redacted>\"\n};\ntry {\nconst res = await fetch(smtp_api, {\nmethod: 'POST',\nheaders: { 'Content-Type': 'application/json' },\nbody: JSON.stringify(data)\n});\nif (res.ok) {\nconst responseData = await res.json();\nalert(\"Successfully submitted. We'll call you.\");\nreturn true;\n} else {\nalert(\"Something went wrong\");\n}\n} catch (error) {\nalert(\"Something went wrong, please try again\");\n}\nreturn false;\n}\nIf you have even a basic sense of security, the problem jumps out: The entire email API endpoint and parameters are exposed in client side code. That means anyone can send arbitrary requests to the backend without using the website at all.\nProof of Concept (redacted)⌗\nThis trivial PoC demonstrates why exposing a mail-sending API client-side is risky:\ncurl -X POST \"https://redacted.example/send-email\" \\\n-H \"Content-Type: application/json\" \\\n-d '{\n\"name\": \"Test User\",\n\"email\": \"[email protected]\",\n\"number\": \"1234567890\",\n\"country_code\": \"+91\",\n\"company_email\": \"[email protected]\",\n\"project_name\": \"Redacted Project\"\n}'\nNote: All identifiers and endpoints are redacted. The goal is to highlight the pattern, not to point at a live system.\nReporting the Bug I reported it to the hosting provider; since the vulnerable app was third-party code hosted on their platform, it was understandably out of scope.\nThe real risk here isn’t theoretical:\nAttackers could spam arbitrary email addresses.\nThey could target the app’s customers with convincing messages.\nThey could target the organization itself by spoofing trusted senders. The Bigger Point\nThe takeaway isn’t about bug bounties, it is baout knowing that LLM generated code is not security aware. LLMs do NOT understand business risk and If you ship AI written examples as it is you may be exposing real attack surface.\nLLMs are helpful assistants but they are NOT really understand the need. Keep humans in the loop for threat modeling, abuse cases thinking and defense design. Else you might end up shipping an app that can be abused with a single curl command.", "url": "https://wpnews.pro/news/why-relying-on-llms-for-code-can-be-a-security-nightmare", "canonical_source": "https://blog.himanshuanand.com/2025/08/why-relying-on-llms-for-code-can-be-a-security-nightmare/", "published_at": "2025-08-22 08:00:00+00:00", "updated_at": "2026-05-24 02:38:33.580851+00:00", "lang": "en", "topics": ["large-language-models", "cybersecurity", "developer-tools"], "entities": ["Coinbase", "HackerOne"], "alternates": {"html": "https://wpnews.pro/news/why-relying-on-llms-for-code-can-be-a-security-nightmare", "markdown": "https://wpnews.pro/news/why-relying-on-llms-for-code-can-be-a-security-nightmare.md", "text": "https://wpnews.pro/news/why-relying-on-llms-for-code-can-be-a-security-nightmare.txt", "jsonld": "https://wpnews.pro/news/why-relying-on-llms-for-code-can-be-a-security-nightmare.jsonld"}}