This is part 3 of the series "Protect Your Own Service Yourself — Self-Defense Security in the Age of AI." Do the checks you can yourself; when it's hard, EarthLink Network can run the assessment for you, and fix the gaps you can't fix on your own. Series contents Anyone can attack now, with AI — ch
eck your own site first What attackers now hold: when AI runs the attack itself Before you start: the whole picture of checking, and how to test without breaking production (this article) Testing from the outside (1): setup and the first run (coming soon) Testing from the outside (2): attack tools and legitimate scanners find different things (coming soon) Checking the code you wrote: static analysis (coming soon) Checking the parts you depend on: libraries and packages (coming soon) Checking your infrastructure settings for gaps (coming soon) Collect every result in one place and decide what to fix first (coming soon) How to fix what you found: three ways (coming soon) When you cannot fix it yourself: a maintenance contract (coming soon) Keep checking automatically every day — from building to protecting (coming soon) Starting your app on your own laptop does not mean you are safely away from production. In our own service, promptflow, the setup notes say it plainly: even on localhost, the app calls live DynamoDB and S3. In the code today, start it without table or bucket names and every table and the file storage fall back to the production ones; even with the staging values, the file storage is shared with production. Point an attack tool at "the local copy" and its requests can read and write real customer data.
That is why this part comes before any hands-on work. Here are the four rules we follow: Never point a check at production. Use a copy of the code, or a separate environment built the same way. Do not overdo it.
Put hard limits on scope, number of rounds, spend, and time, and let a machine enforce them. Read your cloud provider's rules for security testing before you start. Test only what you own. This part also lays out every kind of check the series covers, which part covers each one, and what you can do for free versus what a paid plan changes.
The short answer on cost: our daily checks run on three free, open-source tools. Between June 26 and October 23, 2026, that daily check ran on schedule 115 times. Here is the actual file from our repository. The local setup needs real AWS keys because the API calls the live database (DynamoDB) and storage (S3).
Which database it hits depends on which values you start it with. Started without table or bucket names, every table and the file storage are the production ones, and .env.local holds live production secrets. If we had run an outside-in scanner against our "local" app started that way, the scanner's requests would have landed in production data. So before you test anything, open your own setup files and find out where your local and development environments actually connect.
"It runs on my machine" tells you where the code runs, not where the data lives. We picked two targets that cannot touch production. A copy of the source code. Checks that read code do not need the app to run at all.
In part 4, we point an AI tool (Strix) at a copy of our login code. A closed environment for outside-in tests. For the tools that send real requests, we copied just our login code and ran it in containers that nothing outside can reach (no port is published), with a simple stand-in for the real login server. The target is reachable only from inside that private network.
Nothing can come in from outside, though outbound traffic is not blocked. The attacking tool does go out to the internet, because it has to call the AI model, but the only host it is allowed to test is the target. Nothing in this setup connects to our production database or our real login system. To be precise about what this is not: it is not a full development environment built like production.
Because the login server is a lenient stand-in, some of what the attack tool reported turned out to be the stand-in's behavior, not a flaw in our product. Part 5 shows that as it happened. Testing a full production-like environment from the outside is something we have not done yet. Even against a copy, put limits on how much you send.
Do not rely on someone watching the screen. People look away. Here is the script that drives the attacking tool in our closed environment. One allowed target.
If the AI tries to test a different host, the script refuses. AI models sometimes misread instructions, so asking nicely is not enough. The machine blocks it. A spending cap. We set $2. The run actually spent $0.0331. A round cap. The AI may decide its next move at most 24 times.
The other tools get the same treatment. In part 4, Strix runs with a $1.00 budget flag on the command line; it finished at $0.45. StackHawk's config caps crawling at 5 minutes and scanning at 15. Narrowing scope is a cap too: promptflow exposes 231 API entry points, and our first deep check covered only the 4 that handle login.
Even a development environment, if it lives in the cloud, sends its traffic through the provider's infrastructure. The big providers publish what customers may test. Two things stand out. First, request flooding is prohibited even against your own resources.
A check that fires thousands of login attempts is exactly that. Caps protect your service and keep you inside the provider's rules. Second, check whether your service is on the list at all. promptflow is served through AWS Amplify, and as of October 23, 2026, Amplify does not appear in the Permitted Services list.
Testing our public environment from the outside would mean talking to AWS Support first. Running our checks against a closed copy kept us clear of that question. The rules for the other large providers: AWS: Permitted services can be tested without prior approval. Flooding and DoS are prohibited; simulating DDoS has its own process (AWS penetration testing policy).
Google Cloud: No need to notify Google before testing your own projects, as long as you follow the Acceptable Use Policy and do not affect other customers (Google Cloud security FAQ). Microsoft Azure: No prior approval or notification, but you must follow Microsoft's Rules of Engagement (Azure penetration testing). On any other host or cloud, ask your provider directly whether security testing is allowed and whether notice is required. Testing systems you do not own is illegal, and it is the opposite of what this series is about.
The trap is the services your app calls: payments, email delivery, sign-in providers. Your app talks to them, but they are not yours. When you run an outside-in tool, make sure its requests cannot reach those third parties. That is the other reason the script above allows exactly one host.
To judge the risk from what you depend on, you do not scan the vendor. You compare your list of dependencies against published vulnerability data. That is part 7 (dependencies) and part 9 (bringing results together). There are two broad ways to check a service.
One sends real requests to a running app from the outside, the way an attacker would (dynamic testing, or DAST). The other looks at what is inside without running it (static checks). Static checks split further by what they look at. Kind of check What it finds Tools we use as examples Part Testing from the outside Weak points in the running app's entry points: missing protective response headers, login handling mistakes ZAP (free), StackHawk (paid, with a trial), HexStrike (an attacker's tool) 4, 5 Checking the code you wrote Risky code patterns, keys or passwords left in the code Semgrep, Gitleaks (both free), Strix (an AI that reads code) 4, 6 Checking the parts you depend on Libraries with known vulnerabilities Trivy (free), Dependabot (free on GitHub) 7 Checking infrastructure settings Server and cloud settings with gaps: too much exposed, missing locks Trivy's config scanner, Checkov, Prowler (all free) 8 Bringing results together Merge duplicates, sort by severity, record when each was fixed DefectDojo (free) 9 Fixing, or getting help Fix it yourself, have AI fix it, or hand it to someone who can — 10, 11 Running it every day Notice a newly disclosed vulnerability the next morning GitHub Actions on a schedule 12 AI is not a separate row.
There is an AI-driven version inside each approach: HexStrike lets an AI drive outside-in testing, and Strix lets an AI read code. Part 4 runs both. The outside view and the inside view find different things. That is not theory; it is what happened on a copy of our own login code.
Part 5 lines the results up. In one sentence: the outside-in tool found missing protective headers on responses, while the code-reading tool found that logging out left the server-side session alive and that the audit log recorded "who did it" from a token it had never verified. Almost nothing overlapped. Inside checks also have an advantage for beginners: they never run the app, so they never touch production or a development environment.
It is the safest place to start. Our promptflow repository, which is private on GitHub, runs a security check once a day, in the morning Japan time, using GitHub Actions. Trivy compares the libraries we use against known vulnerabilities. That vulnerability data is updated daily, so code that has not changed can still turn up a new problem tomorrow.
That is the reason to run it every day. Gitleaks looks for keys and passwords left in the code. Semgrep checks the code we wrote against a security rule set (p/security-audit). All three are free and open source: Trivy under Apache-2.0, Gitleaks under MIT, and Semgrep's engine under LGPL-2.1.
Results go to DefectDojo, where we collect everything (part 9), and a chat alert fires only when a new Critical or High finding appears. Is it actually running? Here is the run history. The first run was on June 26, 2026.
By October 23, it had run on schedule 115 times: 114 succeeded and 1 failed. (Someone also started it by hand once.) Most runs finish in a minute or two. Nobody does anything by hand each morning. We also use Dependabot, which opens pull requests when a library we use has a newer version.
To be clear: in our setup, Dependabot runs monthly, not daily. The pull requests it actually opened (part 7 shows the list) mostly arrive at the start of the month: 69 npm updates on October 1 and 58 on September 1. In August, the grouped npm request was opened four times (on the 1st, 4th, 7th and 11th, each replacing the last), and the October 1 and September 1 batches were closed without merging. Getting a pull request is not the same as taking the update; why we closed them is in part 10, using the October 1 batch as the example.
The daily vulnerability check is Trivy's job; the monthly version bumps are Dependabot's. Dependabot also has free alerts that flag a vulnerable dependency right away, and Dependabot security updates, which open fix requests for vulnerable dependencies automatically. Until we checked for this article, we had neither turned on (GitHub's API said both were off on October 23, 2026). We turned both on that same evening.
Now a vulnerable dependency gets a fix request right away instead of waiting for the monthly run, and a few arrived that same night. What changes with them on is something part 7 will look at. We opened each vendor's official pricing page on October 23, 2026 and captured it. Prices change, so check the current page before you buy.
ZAP, an outside-in scanner, is free and open source, permanently. StackHawk, another outside-in scanner, has no permanent free plan. When part 5 shows StackHawk results, they come from that 14-day trial. Keeping it would cost $10 per user per month.
For checking your own code, GitHub's CodeQL is the well-known option, but its free tier covers public repositories only. promptflow is a private repository, so CodeQL would cost $30 per active committer per month on top of a GitHub Team or Enterprise plan. GitHub's API confirms we do not have it: "Code Security must be enabled for this repository to use code scanning." We do not use CodeQL; for checking our own code, we run the free Semgrep every day. Semgrep has its own free and paid tiers. There are two free ways to use Semgrep: the command-line Community Edition with no login, and the Free Edition, where you sign up for Semgrep's service. Our daily check uses the first. According to Semgrep, the Community Edition analyzes within a single function or file and does not follow data across files. The Free Edition adds cross-file analysis for up to 10 contributors and 10 repositories.
Across the tools, the difference between free and paid comes down to five patterns: The checks themselves are mostly free. ZAP, Trivy, Gitleaks, Semgrep, Checkov, Prowler, and DefectDojo are all open source. Where a paid version exists, it adds things like scheduled scans, a central dashboard, ticketing and chat integrations, and support. Prowler's paid service is one example, per its own pricing page.
Public repositories are free; private ones cost money. That is how GitHub's CodeQL and secret protection work: $30 and $19 per active committer per month on a private repository. Dependabot, though, is available on the Free plan. Free tiers are capped by volume. Semgrep's Free Edition stops at 10 contributors and 10 repositories. Some products charge for depth. Semgrep's command-line edition stays inside one file; the registered and paid tiers follow data across files. Some products have no permanent free plan.
StackHawk offers a 14-day trial only. When a page says "free," check whether it means a trial or forever. The AI tools are free software too: Strix is Apache-2.0 and HexStrike is MIT. What you pay for is the AI model running inside them, billed by whichever AI provider you choose.
With a low-cost model (DeepSeek), one Strix run cost us $0.45 and one HexStrike run cost $0.0331. Checking all 231 entry points deeply in one go would take real money and time. You have three options: Run the free inside checks across everything first. Tools like Trivy, Gitleaks, and Semgrep cost nothing to run, so they can cover the whole codebase every day.
That is our daily check. Narrow the scope and go deep one area at a time. Have an AI dig into one group of entry points at a time. Our Strix run on the 4 login entry points cost $0.45.
Pay to go deep everywhere at once. If you have many entry points and want them all covered deeply now, budget for it. For a small team, start with the first and move on to the second. Do not aim for perfect on day one.
Compared with what a data leak costs in apologies to customers and downtime, this is cheap insurance. Deciding up front how much you will spend on checks is part of running a service now. You checked your setup files to see which database and storage your local and development environments actually use. You chose a target that is not production: a separate environment, or a copy of the code.
Your outside-in tool has exactly one allowed target, and a machine refuses anything else. You set caps on spend, rounds, and time, in the tool's config or on the command line. You read your cloud provider's testing rules and checked whether the services you use are on the permitted list (and, if one is not, asked the provider first). Nothing you do not own, such as a payment or sign-in provider, is inside the test's reach.
Part 4 is where the hands-on work starts: testing from the outside, part one. We take the same class of AI security tool an attacker would use and run it against a copy of our own service. From installing it, to connecting a low-cost AI model, to running it with a spending cap, to the first results, with every command and screen shown. In this series, in an age when attack tools are free and one click away for anyone, I show step by step how to check your own service yourself and close the gaps before someone else finds them.
The target is always our own service; the series does not cover attacking anyone else's site. Our products are listed at https://www.eln.ne.jp/products EarthLink Network builds its own products to run the whole company on AI — each one born from a need along the way. See the full index of what we build: → The 18 products EarthLink Network builds in-house For more about the company and each product, visit www.eln.ne.jp.
Key Takeaways #
- •In our own service, promptflow, the setup notes say it plainly: even on localhost, the app calls live DynamoDB and S3.
- •This story was reported by Dev.to , covering developments in thedev space.
- •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.
📖 Continue reading the full article: