{"slug": "block-env-and-config-file-probes-in-express-and-fastify", "title": "Block .env and config-file probes in Express and Fastify", "summary": "A developer published a guide for Node.js APIs showing how to block credential-hunting probes for files like .env, .git/config, .aws/credentials and AI coding tool configs such as .mcp.json and .continue/config.json. The approach verifies sensitive paths return 404 or 403 in production, serves static files only from a public folder with dotfiles denied, and mounts a regex-based probe guard before body parsing, auth or database work in Express and Fastify. Hits are logged by IP, user agent and path, with repeat offenders banned after crossing a threshold, since user agents like CCBot cannot be trusted.", "body_md": "Open the 404 log of any public API and you will find requests for files that were never meant to be served: `/.env`, `/.git/config`, `/.aws/credentials`. The list has grown. Scanners now ask for the config files of AI coding tools too, because those files hold API keys and MCP server tokens.\n\nThis post covers three things for a Node API: confirming none of those paths can return anything, answering them with a fast 404 before your app does any work, and counting the IPs behind them so repeat offenders get blocked.\n\nSlobodan Manic [published 24 hours of his own site's logs](https://searchenginejournal.com/the-biggest-ai-crawler-on-my-website-was-hunting-for-credentials/585159) (Search Engine Journal, Aug 14). About 100 credential-hunting paths drew 1,028 requests and sent zero referrals. A few of the counts:\n\n| Path | Requests | \n|---|---|\n| `/.ssh/known_hosts` | 42 | \n| `/.mcp.json` | 30 | \n| `/.env.production` | 29 | \n| `/.continue/config.json` | 24 | \n\nMost of it carried a crawler user agent (CCBot), and the author says he couldn't verify whether it was the real crawler or someone borrowing the name. That matters for the fix: a user agent is whatever the client says it is, so don't build your blocking on it.\n\nA fast 404 is pointless if the file is being served. Run the paths against production, and print the size as well as the status, because an SPA fallback can return `200` with your `index.html` for everything:\n\n```\nfor p in /.env /.env.production /.git/config /.aws/credentials \\\n         /.ssh/known_hosts /.mcp.json /.continue/config.json; do\n  curl -s -o /dev/null -w \"%{http_code} %{size_download} $p\\n\" \"https://api.example.com$p\"\ndone\n```\n\nAnything other than a 404 or 403 gets a closer look. Then check how static files are served:\n\n`public` folder, never at the project root or `process.cwd()`. That one line is how `.env` and `.git/` end up on the internet.`express.static('public', { dotfiles: 'deny' })``@fastify/static`, leave `serveDotFiles` off (it defaults to `false`).\nOnce nothing can leak, make the probes cheap. They should get a 404 before session lookup, body parsing, auth or a database query. Match on any path segment, since scanners also try `/api/.env` and `/app/.git/config`, and decode the path first so `/%2eenv` doesn't slip past:\n\n``` js\n// probe-guard.js\nexport const PROBE =\n  /(?:^|\\/)(?:\\.env(?:\\.[\\w-]{1,32})?|\\.git|\\.aws|\\.ssh|\\.mcp\\.json|\\.continue|\\.cursor|\\.vscode|\\.idea)(?:\\/|$)/i;\n\nexport function decodedPath(url) {\n  const path = url.split('?')[0];\n  try { return decodeURIComponent(path); } catch { return path; }\n}\n```\n\nExpress: mount it first, above everything else.\n\n``` python\nimport express from 'express';\nimport { PROBE, decodedPath } from './probe-guard.js';\nimport { recordProbe, banCheck } from './probe-counter.js';\n\nconst app = express();\napp.use(banCheck);\napp.use((req, res, next) => {\n  if (!PROBE.test(decodedPath(req.originalUrl))) return next();\n  recordProbe(req.ip, req.get('user-agent'), req.originalUrl);\n  res.status(404).set('Cache-Control', 'no-store').end();\n});\napp.use(express.json());\n// ...auth, routes\n```\n\nFastify: [`onRequest`](https://fastify.dev/docs/latest/Reference/Hooks/) runs before the body is parsed, which is where this belongs.\n\n``` js\nfastify.addHook('onRequest', async (request, reply) => {\n  if (!PROBE.test(decodedPath(request.url))) return;\n  recordProbe(request.ip, request.headers['user-agent'], request.url);\n  reply.code(404).header('cache-control', 'no-store').send();\n  return reply;\n});\n```\n\n`/.well-known/` doesn't match, so ACME challenges and `security.txt` keep working.\n\nOne probe is background noise. Twenty from the same address in a few minutes is someone working through a list. Log each hit with the IP, user agent and path, and deny an address that crosses a threshold:\n\n```\n// probe-counter.js (per process; use Redis or your edge for several instances)\nconst WINDOW_MS = 10 * 60_000, LIMIT = 5, BAN_MS = 60 * 60_000;\nconst hits = new Map();\nconst banned = new Map();\n\nexport function recordProbe(ip, ua, path) {\n  const now = Date.now();\n  const recent = (hits.get(ip) ?? []).filter((t) => now - t < WINDOW_MS);\n  recent.push(now);\n  hits.set(ip, recent);\n  console.warn(JSON.stringify({ msg: 'probe', ip, ua, path, count: recent.length }));\n  if (recent.length > LIMIT) banned.set(ip, now + BAN_MS);\n}\n\nexport function banCheck(req, res, next) {\n  const until = banned.get(req.ip);\n  if (until && until > Date.now()) return res.status(403).end();\n  if (until) banned.delete(req.ip);\n  next();\n}\n```\n\nThe same `banCheck` logic works as a Fastify `onRequest` hook. Before you rely on it, check three things:\n\n`req.ip` is only the real client when `trust proxy` names your actual proxies. Trust every hop and a scanner can write any address into `X-Forwarded-For`, including one of your customers'.` hits` on a timer, or the map grows with every scanner that has ever visited.\nThe paths change. AI tool configs are a recent addition, and the next tool's config folder will show up in your logs before it shows up in a blog post. Once a month, group your 404s by path, look at the top 50, and add anything that is clearly a credential or config file. Good candidates today: `.env` variants, `.git/`, `.aws/credentials`, `.ssh/`, `.mcp.json`, and editor or agent folders such as `.vscode/`, `.cursor/` and `.continue/`.\n\nApiTraffic's firewall runs inside your Node app through the SDK. Its managed rule set has Scanners & Probes and Sensitive Path Probing categories, and a custom rule can block a path regex on every server; policy changes reach connected servers without a deploy. Behavioral protection scores each IP over a window on signals that include 404 responses and requests to sensitive paths, and in Enforcing mode it pushes a temporary deny to every server running your app, so the counting above works across instances without Redis.\n\nSee what's hitting your API: [apitraffic.io](https://apitraffic.io/?utm_source=blog&utm_medium=content&utm_campaign=ai-era&utm_content=bf-20261106-block-env-probes-express)", "url": "https://wpnews.pro/news/block-env-and-config-file-probes-in-express-and-fastify", "canonical_source": "https://dev.to/apitraffic/block-env-and-config-file-probes-in-express-and-fastify-2l6g", "published_at": "2026-11-06 12:18:00+00:00", "updated_at": "2026-11-06 12:24:22.271419+00:00", "lang": "en", "topics": ["ai-crawlers", "ai-tools", "developer-tools"], "entities": ["Express", "Fastify", "Slobodan Manic", "Search Engine Journal", "CCBot"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/block-env-and-config-file-probes-in-express-and-fastify", "markdown": "https://wpnews.pro/news/block-env-and-config-file-probes-in-express-and-fastify.md", "text": "https://wpnews.pro/news/block-env-and-config-file-probes-in-express-and-fastify.txt", "jsonld": "https://wpnews.pro/news/block-env-and-config-file-probes-in-express-and-fastify.jsonld"}}