Writing More Secure Code with LLMs: Why "Make No Mistakes" Falls Short

A Monad Foundation engineering experiment found that adding a real threat model to an LLM coding prompt cut validated security findings by about 43%, from 21 to 12, while a generic OWASP-style "be secure" checklist barely moved the total (20) and doubled high-severity bugs from 2 to 4. Author Kristov Atlas ran a single URL-unfurler build task once per prompt variant with the same model, then had four independent AI reviewers scan each result with a separate validation pass over every finding. The team published the results to help Monad ecosystem builders ship more secure AI-generated code.

All posts https://monad.xyz/blog Writing More Secure Code with LLMs: Why "Make No Mistakes" Falls Short Kristov Atlas @kristovatlas https://x.com/kristovatlas - Published on - · 18 min read What actually makes an AI write safer code, measured on one build task with validated vulnerability counts. First in a multi-part series from the Monad Foundation engineering team. Our engineering team has two important callings: to create projects that fulfill an unmet niche in the Monad ecosystem, and to encourage solid practices among other Monad builders. Increasingly, we do that work with the help of LLMs. We've noticed that existing research on writing secure code has focused largely on bug detection. To shift attention "left" in the software lifecycle, we've been researching how to guide LLMs to write better code from the start. Here's a tempting assumption: tell an AI coding assistant to be secure and not make mistakes, and it will write secure code. We tested that. We held the task and the model fixed, wrote three different prompts, turned a panel of independent AI security reviewers loose on each result, and ran a validation pass over every finding they reported. The generic "be secure" checklist barely moved the total and doubled the high-severity bugs. When that generic approach flopped, we wrote an expert prompt that spelled out a real threat model for that specific app, and it cut total findings by about 43% . The task and the model stayed the same; the only thing we changed was the words in the prompt. We scored each prompt's output against the same task and bucketed the validated findings by severity: | Prompt variant | Critical | High | Medium | Low | Info | Total | |---|---|---|---|---|---|---| | baseline just the feature spec | 0 | 2 | 3 | 10 | 6 | 21 | | expert feature spec + a real threat model | 0 | 2 | 2 | 5 | 3 | 12 | | inline-checklist feature spec + a generic OWASP-style checklist | 0 | 4 | 1 | 8 | 7 | 20 | Each run was a single generation with no fixing afterward. Four independent AI reviewers scanned the result, and a separate step confirmed or threw out every finding. This is a sample size of one: I ran it once per prompt, so treat the counts as a rough signal and the exact numbers as noisy. The limitations limitations section is blunt about how far these go. A lot of our first drafts come straight out of AI coding assistants. How secure those drafts are decides how much review a feature needs before it ships, so "what makes the model write safer code" is a question we care about for selfish reasons. We're publishing because the other half of our job is helping teams across the ecosystem ship secure code, and this part is easy for you to implement immediately: it works with whatever model you already use, and it ends with a workflow you can paste into your next feature. None of it is blockchain-specific. If your team writes code with an AI assistant, it's for you. The task was a URL unfurler: the thing that turns a pasted link into a preview card in Slack or Discord. A user sends a URL, your server fetches it, parses the returned HTML for the title and OpenGraph tags, and hands back a little JSON card. Step 2 is the whole security problem. Your server makes an outbound request to a URL that a stranger picked. That's SSRF, Server-Side Request Forgery, and it bites because your server sits somewhere far more privileged than a random person on the internet. From inside a cloud VPC it can reach things the attacker can't: - Cloud metadata endpoints. Every major cloud exposes a magic internal address, 169.254.169.254 , that hands a machine its own details, including temporary credentials for the cloud account. Get the unfurler to fetch it and the server may hand back the keys. This is the same class of bug behind the 2019 Capital One breach, where an SSRF reached the EC2 metadata endpoint and walked off with data on more than 100 million people. - Internal-only services like admin dashboards, databases, and the Jenkins box: firewalled from the internet, wide open from inside. - Loopback and private ranges such as 127.0.0.1 , 10.x , and 192.168.x . This is the confused-deputy problem. Your server holds network access the attacker doesn't have, and the attacker talks it into spending that access on their behalf. Any feature where a server fetches a resource the user named is an SSRF surface: webhooks, PDF generators, image proxies, "import from URL" fields. The unfurler is just a clean example of the class. Baseline validated almost nothing. It checked that the URL started with http or https , then let the HTTP client chase redirects wherever they led. Point it at the metadata endpoint and off it goes, no cleverness required. Expert wrote real defenses into the first draft. It resolved the hostname and rejected private, loopback, and link-local addresses; pinned the connection to the address it had already validated, so the target couldn't swap after the check; re-checked every redirect hop; capped response size and time; and bound to localhost by default. Total findings dropped from 21 to 12. Two high-severity findings survived, but they're narrow bypasses of working defenses that take a specialist to reach. Baseline's high-severity bug took a single curl . Inline-checklist got a sensible generic checklist: validate inputs, cap resources, handle errors. It cut the total by one and doubled the high-severity count. A checklist names the categories of defense without saying what actually attacks this app, so the model bolted on SSRF defenses and wired a specific bug into each one: - a DNS rebinding race: it resolved and checked the hostname, then let the HTTP client resolve it again at connect time. An attacker who runs the DNS answers with a safe public IP for the check and 127.0.0.1 for the connection. That's a TOCTOU bug time-of-check to time-of-use , and baseline never had it, because baseline checked nothing. - an IP filter built as a hand-written deny list that lets whole special-use ranges through. - a synchronous DNS call sitting on the async event loop, stalling the entire server. Baseline had a single broad hole. The checklist version swapped it for three narrow ones, each its own exploit. It bolted three shiny new deadbolts onto a door still hanging off its hinges. A half-finished defense can be worse than none, because it looks done and hands the attacker something specific to work on. What moved the numbers was a threat model tied to this app's real data flows. The word "security" in the prompt did nothing by itself. The expert prompt came from a security-aware author and still shipped bugs. They aren't random. Three traps caused them, and you'll hit the same three far outside URL unfurlers, so learn them by name. Trap 1: enumerate a list where a rule would do. The prompt spelled out the bad IP ranges to block, and the model blocked exactly those, no more. It missed 100.64.0.0/10 , the CGNAT range that Tailscale and Kubernetes overlay networks lean on and that Python's is private won't flag either . The fix a specialist reaches for is to test the underlying property: not ip.is global asks whether an address can route on the public internet, and it catches CGNAT plus every range invented after you shipped. Any hand-maintained list, of file extensions or MIME types or domains or headers, misses the entry nobody thought of. Prefer one predicate that says what you actually mean. Trap 2: compare two values that went through different parsers. The code pinned the connection to the validated IP, but only when the request's hostname matched the one it had validated. Those two hostnames came out of two different URL parsers, which disagree on internationalized domain names bücher.example versus its punycode form xn--bcher-kva.example . On those names the check quietly failed, the pin got skipped, and the rebinding hole it was built to close yawned back open. A check that errs toward allowing is failing open, and open is the wrong default. When you validate a value and later compare it to the one in use, push both through the same normalizer first. The same mismatch sits behind a lot of path-traversal and Unicode auth-bypass bugs. Trap 3: validate inputs and forget outputs. The threat model covered attacker → your service → internal target. It skipped attacker → your service → the value you return → your consumer → internal target. When a page's og:image pointed at an internal-only host, the service passed it straight through, and any downstream renderer that fetches that image becomes the confused deputy instead. The code even flagged this out of scope, which fit its too-narrow threat model. Every attacker-influenced value you return, store, or log is live input to whatever eats it next. Most stored-XSS and second-order injection lives right here. You can't reuse the expert prompt as a template. Its power was SSRF-specific, and pasted into a CSV importer it's just noise. The move that travels is to make the model build the threat model itself from a plain description of your system. You supply the procedure; the model fills in the domain. Do this in the planning phase, before any implementation code. In Claude Code, that's literally plan mode. Step 1: describe the system in plain functional terms. Endpoints, inputs, outputs, data stores, external calls, and who calls it from where. Leave security out for now. That keeps the description honest and forces the next step to reason, so it can't just echo security notes you already wrote. Step 2: run the security-engineer pass. Paste the prompt below right after your description. It says nothing about URLs or IPs; the model fills in the domain from what you wrote. Now switch roles. You are a senior application-security engineer doing a pre-implementation design review. Do NOT write implementation code yet. Work only from the system description above. Produce three sections: A APP-CLASS RISK PROFILE Classify what kind of system this is in security terms e.g. "a service that fetches user-supplied resources," "a service that renders user-controlled content," "a service that runs user-influenced queries / commands," "a multi-tenant data store" . For that class, list the vulnerability categories that TYPICALLY bite this kind of app. For each, tie it to a SPECIFIC data flow in the description above and say why it applies here — not a generic checklist. If a category does not apply, say so and why. B THREAT MODEL - Trust boundaries: every point where data crosses from untrusted to trusted. - Attacker-controlled inputs: enumerate every value an attacker can influence — direct inputs AND indirect ones responses from upstream services, redirects, resolved addresses/DNS, values you stored earlier and read back, filenames, headers . - Privileged position & assets: what can this code reach, do, or hold that an external attacker cannot reach directly internal network, credentials, filesystem, other tenants' data, ability to send mail / spend money ? - Attacker goals: the 3–5 highest-value things an attacker would attempt, each mapped to the data flow that enables it. C DEFENSIVE-PROGRAMMING STANDARDS the definition of done For each risk in A/B, state the defense as an INVARIANT the code must uphold — an outcome, not a step. "No operation may reach a resource the caller could not reach directly," not "call validate ip ." Then commit to these general standards, which catch the bugs that survive even careful security work: 1. PRIMITIVES OVER LISTS. Prefer a single library/stdlib predicate that expresses the security intent over a hand-maintained allow/deny list — a list silently misses the entry no one thought of. State what each check covers and what it does not. 2. NORMALIZE BEFORE YOU COMPARE. If you validate a value and later compare it or a transform of it against a value in use, both sides must be produced by the SAME parser/normalizer. Mismatched parsing between validation and use is a classic bypass. 3. VALIDATE OUTPUTS LIKE INPUTS. Any attacker-influenced value you return, store, log, or pass on — assume a downstream consumer will act on it, and validate it as if it were an input to that consumer. 4. FAIL CLOSED. On any parse error, ambiguity, or unexpected state, deny rather than allow. Set explicit timeouts, size caps, and iteration limits. Choose safe defaults least exposure, least privilege . 5. NAME YOUR NON-GOALS. State what this component explicitly does NOT defend against, so the next person doesn't assume it does. Treat section C as the acceptance criteria: the implementation is complete only when every invariant in C is demonstrably upheld. Once I approve this, implement against it. Step 3: build against section C, then check against it. Hold the assistant to section C as the definition of done, and make it show you where each invariant is enforced. That's the step that turns a threat model into working defenses. Skip it and section C becomes a doc nobody reads. This hands the model a real threat model, which is what won it for the expert prompt, and it derives that model from your system, which is what the checklist never had. Standards 1 through 3 are the three traps from earlier, written as rules you keep. For code that matters, add one step. Take section C to a different model family generate with one, audit with another and tell it to attack the standard: what does this miss? A different training prior tends to catch what the first model structurally couldn't. Don't grade your own homework on the code that can get you popped. Most research on the security of AI-generated code measures the problem. "Asleep at the Keyboard?" https://arxiv.org/abs/2108.09293 Pearce et al., IEEE S&P 2022 found about 40% of GitHub Copilot completions in security-relevant scenarios came out vulnerable, a 2021 number that newer models beat. Since then it's mostly benchmarks: CyberSecEval https://arxiv.org/abs/2312.04724 Meta, 2023 , SeCodePLT https://arxiv.org/abs/2410.11096 NeurIPS 2025 , BaxBench https://arxiv.org/abs/2502.11844 ICML 2025 , SecRepoBench https://arxiv.org/abs/2504.21205 2025-26 , SecureAgentBench https://arxiv.org/abs/2509.22097 ACL 2026 . They measure how insecure the code is, with static analysis, test oracles, and live exploits. Getting the model to write safer code up front gets a lot less attention. Tony et al. https://arxiv.org/abs/2407.07064 ACM TOSEM 2025 is the one dedicated, peer-reviewed study of prompting for secure code generation I found, and several of those benchmarks tack on a prompt-condition experiment as a side ablation. The pattern in them is consistent: security prompts help a lot on small self-contained tasks SecRepoBench measured +19 points on one and much less at repository scale +1.6 points on the same study's real repos , and BaxBench calls its "tell the model the exact weakness to avoid" prompt an unrealistic upper bound. A separate line retrains or fine-tunes models for this SVEN, SafeCoder, PromSec ; I've set that aside, because the question here is what you can do with the model you already have, no training run required. This work puts prompt content in the foreground and pins the task, model, and workflow in place. A later article in the series takes the question I couldn't find anyone answering head-to-head: does security guidance in the first prompt beat generating code and then repairing it with a scanner, holding the workflow constant. Read the numbers as a rough signal, with these caveats: - Sample size is one. One task, one model, one run per prompt. Model output wobbles between runs, so a rerun could land somewhere else. I'd read the 43% as "clearly fewer" and leave it there. - The graders are AIs too. They have their own blind spots, and some share a model family with the code generator, so they can agree or miss together. The obvious gaps, like baseline's wide-open SSRF against expert's closed one, are the part I trust. I trust the exact counts a lot less. - Favorable terrain. The task is small and self-contained, which is exactly where the published ablations show prompts helping most. At repository scale the same work says the effect shrinks, so expect a smaller win in a big codebase. - Prevention has a ceiling. Prompting clears the broad, directly exploitable bugs. It won't conjure knowledge the model lacks CGNAT or guarantee a clean implementation of a defense the parser mismatch . Keep your reviews and your tests. - The workflow up top is an extrapolation. It stitches together what won for the expert prompt and fixes for what it missed, and I haven't run it through the experiment yet. Use it as a strong default until I do. - A threat model beats a checklist. Name the asset an attacker wants and the data flow that reaches it. - A half-built defense can be worse than none. If you add a defense, finish it. - Check three traps on every feature: a list where a rule would do, two values compared after different parsers, and outputs that leave the building unchecked. - Fail closed. Choose safe defaults. Say what you are not defending against. - Plan first: describe the system, run the security-engineer pass, build to the standards, and for high-stakes code get a second model family to attack them. A later article in the series runs the two workflows head to head: security guidance in the first prompt against generate-then-scan-and-fix, same task, same scoring. This post is published by the Monad Foundation for general informational and educational purposes only. It describes a single, limited experiment and does not constitute security, engineering, legal, or other professional advice. No representation or warranty is given as to the accuracy, completeness, or fitness for any purpose of the techniques, prompts, or findings described. Readers remain solely responsible for the security of their own systems and should obtain independent professional review before relying on this material. Third party names, products, and marks are the property of their respective owners and are used for identification purposes only; their use does not imply any affiliation with, or endorsement by or of, the Monad Foundation.