{"slug": "who-handles-your-security-reviews", "title": "Who Handles Your Security Reviews?", "summary": "Open-source software security reviews increasingly rely on LLMs, but these tools both find and create vulnerabilities, according to a developer's analysis. The author recommends making security review a habit, using specialized LLM reviewers, scanning existing code, and seeking help from ecosystem programs or human researchers. Specific advice includes avoiding proof-of-concept requests on certain Claude models and considering Anthropic's Cyber Verification Program for security research.", "body_md": "# Who Handles Your Security Reviews?\n\n## Table of Contents\n\n[Door handle designed by Jasper Morrison](https://commons.wikimedia.org/wiki/File:Door_handle_designed_by_Jasper_Morrison.jpg) by JT at JMLtd, licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/).\n\nI’ve had a closer look at what’s happening in the world of OSS security\nthese past few months, and I’ve seen firsthand where vulnerabilities are turning\nup. What I find particularly surprising is that it’s not just in legacy code.\n[CVEs](https://www.cve.org/) are also turning up in brand new code that’s being uploaded right now.\nJust because we now have more tools at our disposal doesn’t mean that\nsecurity issues are a solved problem.\n\nHow do we solve this problem? Probably via review. But review by whom? Maybe\nyou use an LLM. Or a friend who is interested in security. Possibly your\necosystem already has a [program with engineers you can request reviews from](https://alpha-omega.dev/blog/join-the-fight-building-a-team-of-open-source-security-engineers-in-residence/). It\ncould be all of the above.\n\n## [#](#llms-cut-both-ways)LLMs cut both ways\n\nWhile LLMs are very good at finding exploits, they are also really good at\ncreating them — just like humans. The cynical approach to this says that of\ncourse [vibe-coded](https://en.wikipedia.org/wiki/Vibe_coding) software will be riddled with errors, but to be fair, the vibe coders\nalso have defensive tools at their disposal. They’re either not using them\nconsistently or stuff is just getting through — in the same way that you can\nget a security review from your very annoying and brutally thorough colleague\nand they will likely still have missed something.\n\n## [#](#make-security-review-a-habit)Make security review a habit\n\nIf you use LLMs, one way to improve your odds is to get into the habit of\ninvoking a specialized security reviewer on your new code. Here’s my standard\n[security-review.md](https://github.com/oalders/kitchen-sink/blob/main/commands/security-review.md)\nwhich might give you a starting point if you’re looking for inspiration. At the\nend of the day it’ll come down to what your LLM skills are directing the agent\nto look for but also which model you are using to implement the skill.\n\nDepending on which model you are using, you may want to avoid doing things like\nasking for a [proof of concept](https://en.wikipedia.org/wiki/Proof_of_concept). Certain Claude models have been known to deny\nthe request or even lock you out of your account. (I have that as first-hand\nanecdata). If you’re serious about doing security research and you use Claude,\nyou may want to look at Anthropic’s [Cyber Verification\nProgram](https://support.claude.com/en/articles/14604842-real-time-cyber-safeguards-on-claude-opus-and-sonnet).\n\n## [#](#scan-the-code-you-already-have)Scan the code you already have\n\nAs an interesting exercise, try pointing an LLM scanner at your existing code.\nYou may be surprised at what you find, and maybe you can even ship fixes before\nthe security researchers find them. For example, just a few days ago I found\nsecurity hardening worth doing in\n[diff-lockfiles](https://www.npmjs.com/package/diff-lockfiles) while making\nunrelated changes.\n\n## [#](#ask-for-help)Ask for help\n\nYour ecosystem may already have a program you can turn to:\n\nIf you’re involved with one of these ecosystems or already know a security researcher, ask them to scan your code and send you the results. Or ask a human to review it the old-fashioned way — that has worked for decades and is still a fine choice.\n\nRelated posts:", "url": "https://wpnews.pro/news/who-handles-your-security-reviews", "canonical_source": "https://www.olafalders.com/2026/09/09/who-handles-your-security-reviews/", "published_at": "2026-09-09 00:00:00+00:00", "updated_at": "2026-09-09 04:19:46.341010+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety", "ai-products"], "entities": ["Anthropic", "Claude", "Cyber Verification Program", "diff-lockfiles", "Jasper Morrison"], "alternates": {"html": "https://wpnews.pro/news/who-handles-your-security-reviews", "markdown": "https://wpnews.pro/news/who-handles-your-security-reviews.md", "text": "https://wpnews.pro/news/who-handles-your-security-reviews.txt", "jsonld": "https://wpnews.pro/news/who-handles-your-security-reviews.jsonld"}}