{"slug": "prompt-for-model-to-add-protection-for-your-backend-from-scanners", "title": "Prompt for model to add protection for your backend from scanners", "summary": "A developer published a reusable prompt instructing AI coding assistants to implement a four-layer, fail-open backend defense against automated scanners, log exhaustion, and OTP brute-forcing. The blueprint covers verified reverse-proxy IP resolution, a Redis-backed progressive ban engine with stealth 404s and time-capped bans, an exception filter that keeps 4xx errors out of the database, and distributed OTP attempt counters with a five-failure limit. It emphasizes parameterized thresholds, fail-open behavior under Redis latency, and avoiding hardcoded secrets.", "body_md": "**Purpose:** Hand this document to any AI coding assistant or backend engineer to implement a production-grade, fail-open bot scanner defense, reverse proxy IP resolution, log exhaustion prevention, and OTP brute-force protection system in any backend service.\n\nYou are an expert backend security and infrastructure architect. Your task is to implement a robust, fail-open defense system against automated scanners, web crawlers, log exhaustion attacks, and brute-force credential abuse.\n\nFollow the requirements, algorithms, and implementation blueprints defined below. Maintain fail-open availability: security controls must never degrade uptime or block legitimate users if secondary systems (like Redis) experience latency or outages.\n\nDo not hardcode any sensitive credentials, production ports, or environment-specific secrets. Keep all thresholds and configurations parameterized. Do not use emojis in code comments, responses, or commit messages.\n\n1. **Automated Vulnerability Scanning:** Botnets and scanners (Shodan, Censys, automated crawlers) bombard endpoints searching for sensitive paths (`.env` ,`.git` ,`wp-login.php` ,`phpmyadmin` , backup dumps). Leaving these unmitigated wastes CPU, memory, and database bandwidth.\n2. **The \"Self-Inflicted Proxy DoS\" Trap:** When running behind reverse proxies (Cloudflare, Nginx, container networks), naively blocking`req.ip` without configuring proxy trust bans the reverse proxy itself, creating a 100% outage for all users. Reading headers like`X-Forwarded-For` or`CF-Connecting-IP` directly without verification allows attackers to spoof arbitrary IP addresses.\n3. **Database Log Exhaustion (404 Floods):** Applications that log unhandled exceptions to a database table suffer catastrophic disk I/O and storage bloat when a scanner generates hundreds or thousands of 404 requests in seconds.\n4. **Carrier-Grade NAT (CGNAT) Collateral Damage:** Mobile users share public carrier IP addresses. Permanent IP bans will inevitably block innocent users on the same cell tower or public Wi-Fi. Bans must be progressive and strictly time-capped.\n5. **OTP Credential Brute-Forcing:** 6-digit numeric OTPs have 1,000,000 combinations. Without distributed attempt counters, bots can brute-force codes within their active validity window.\n\nYou must implement four distinct defense layers:\n\n```\n[ Incoming Request ]\n        │\n        ▼\nLayer 1: Reverse Proxy & Real IP Resolution\n  - Trust only verified proxy CIDRs (Cloudflare + Local/Container networks)\n  - Framework right-to-left X-Forwarded-For traversal\n        │\n        ▼\nLayer 2: Scanner Detection & Progressive Redis Ban Engine\n  - Whitelist bypass for legitimate application routes (/api/v1/*, etc.)\n  - Fast regex for scanner probes, dangerous extensions, and malformed URIs\n  - Configurable strike sliding window in Redis (e.g. 3 strikes in 60s)\n  - Stealth 404 responses during probing (do not signal an active WAF)\n  - Progressive ban durations with 24h retention (e.g. 15m -> 30m -> 1h max)\n  - Generic 403 Forbidden for actively banned IPs (no fingerprinting)\n  - Strict fail-open design with command timeout (<= 80ms)\n        │\n        ▼\nLayer 3: Global Exception Filter (Log Exhaustion Guard)\n  - Never write 4xx errors (especially scanner 404s) to the database\n  - 404s logged at debug level only\n  - Other 4xx logged at warning level\n  - 5xx logged at error level and persisted to database/persistent storage\n        │\n        ▼\nLayer 4: Distributed OTP Brute-Force Protection\n  - Redis attempt counter scoped to normalized recipient identifier\n  - TTL matching the OTP expiration window\n  - Hard limit of 5 failed attempts\n  - Automatic database revocation of active OTPs upon threshold breach\n```\n\nWhen generating or deploying this system, strictly enforce the following rules:\n\n1. **Parameterize Timing & Strike Thresholds:** Do not hardcode strike limits (e.g., 3 strikes in 60s) where an attacker reading source code can calibrate scan speed to fly just under the radar. Allow overriding via environment variables.\n2. **Prevent Server Fingerprinting:** Return generic, standard error messages (e.g. standard`403 Forbidden` response) rather than custom signatures that identify the backend technology or internal firewall software.\n3. **Strict Whitelist Boundary:** Never put root`/` , wildcard patterns, or public user-content paths into the safe route bypass list. Only include verified application route prefixes.\n4. **Mandatory Expiration (TTL) on All Cache Keys:** Every key created in Redis (`scan:strike:*` ,`scan:ban:*` ,`scan:offenses:*` ,`auth:otp_attempts:*` ) MUST have an explicit TTL. Never write unbounded keys that can lead to memory exhaustion (OOM DoS).\n5. **No Direct Header Trust:** Under no circumstances should`req.headers['x-forwarded-for']` or`req.headers['cf-connecting-ip']` be read directly. Trust proxy configuration must be used so the framework validates proxy hops.\n6. **No Secret Leaks in Exceptions:** Ensure exception filters sanitize query params, request bodies, and database errors to prevent leaking database structure or environment variables in responses.\n\nEnsure `req.ip` resolves to the authentic remote client IP, preventing both proxy banning and header spoofing.\n\n1. Never read `req.headers['x-forwarded-for']` or`req.headers['cf-connecting-ip']` directly.\n2. Configure framework proxy trust using verified upstream CIDR blocks.\n3. Express/Connect-based frameworks evaluate `X-Forwarded-For` from right to left, stripping trusted hops until the first untrusted IP is reached.\n\nInclude official Cloudflare ranges and private container/bridge networks (adjust if your reverse proxy infrastructure differs):\n\n- **Private / Container Networks:**`'loopback'` ,`'linklocal'` ,`'uniquelocal'`\n- **Cloudflare IPv4:**`173.245.48.0/20` ,`103.21.244.0/22` ,`103.22.200.0/22` ,`103.31.4.0/22` ,`141.101.64.0/18` ,`108.162.192.0/18` ,`190.93.240.0/20` ,`188.114.96.0/20` ,`197.234.240.0/22` ,`198.41.128.0/17` ,`162.158.0.0/15` ,`104.16.0.0/13` ,`104.24.0.0/14` ,`172.64.0.0/13` ,`131.0.72.0/22`\n- **Cloudflare IPv6:**`2400:cb00::/32` ,`2606:4700::/32` ,`2803:f800::/32` ,`2405:b500::/32` ,`2405:8100::/32` ,`2a06:98c0::/29` ,`2c0f:f248::/32`\n\nDetect automated bot probes, increment strike counts in Redis, ban repeat offenders progressively, and return stealth responses.\n\nBefore route handling, check if the client IP exists in Redis:\n\n- Redis key: `scan:ban:<client_ip>`\n- If key exists: immediately return HTTP `403 Forbidden` with a generic message:`\"Forbidden\"` .\n\n- Extract full raw path (`baseUrl + path` ).\n- Attempt `decodeURIComponent(rawPath)` .\n- If decoding throws `URIError` (e.g. invalid byte sequences or malformed percent-encoding like`%c0%ae` ), treat the request immediately as a scanner probe.\n\nDo not evaluate regex or Redis strikes against legitimate application routes. If `decodedPath` starts with any of the configured prefixes, immediately pass to `next()`:\n\n- `'/api/v1'` (or application route prefix)\n- `'/api/docs'` (API documentation)\n- `'/health'` (health-check endpoints)\n- `'/favicon.ico'`\n\nEvaluate non-whitelisted paths against the scanner regex:\n\n```\n/(?:^\\/|\\/)(?:\\.env[^/]*|\\.git[^/]*|\\.svn[^/]*|\\.aws[^/]*|\\.ssh[^/]*|\\.npmrc[^/]*|\\.netrc[^/]*|\\.DS_Store[^/]*|[^/]*_history|wp-admin|wp-login|wp-content|wp-includes|wp-config|phpmyadmin|pma|xmlrpc\\.php|cgi-bin|actuator|eval-stdin\\.php|web\\.config)(?:\\/|$|\\.|\\?)|(?:\\.php|\\.sql|\\.bak|\\.old|\\.orig|\\.save|\\.swp|\\.zip|\\.tar|\\.gz|\\.tgz|\\.bz2|\\.rar|\\.7z)(?:\\?.*)?$/i\n```\n\nIf the pattern does not match and URI is not malformed, pass to `next()`.\n\nWhen a probe is detected:\n\n1. Increment strike counter: `INCR scan:strike:<client_ip>`\n2. If counter equals 1, set window expiry: `EXPIRE scan:strike:<client_ip> <STRIKE_WINDOW_SEC>` (default: 60 seconds).\n3. If strikes < threshold (default: 3): Return HTTP `404 Not Found` . Never return 403 or custom messages during strikes, keeping the trap stealthy.\n4. If strikes >= threshold:\n  - Increment historical offenses: `INCR scan:offenses:<client_ip>`\n  - If offenses equals 1, set retention expiry: `EXPIRE scan:offenses:<client_ip> 86400` (24-hour memory).\n  - Select ban duration based on offense tier:\n    - Offense 1: 900 seconds (15 minutes)\n    - Offense 2: 1800 seconds (30 minutes)\n    - Offense 3+: 3600 seconds (1 hour maximum cap for CGNAT safety)\n  - Store ban: `SET scan:ban:<client_ip> \"1\" EX <ban_duration_sec>`\n  - Delete strike key: `DEL scan:strike:<client_ip>`\n  - Return HTTP `404 Not Found` for the probe request that triggered the ban.\n5. Increment historical offenses: \n\n- Wrap all Redis operations in `try/catch` .\n- If Redis is unavailable or times out, catch the error, log a rate-limited warning (cooldown of at least 30 seconds), and allow the request to proceed (`fail-open` ).\n- Configure Redis client with:\n  - `commandTimeout` : 80ms maximum.\n  - `enableOfflineQueue` :`false` (do not buffer commands when disconnected).\n  - `maxRetriesPerRequest` : 1.\n\nPrevent high-frequency scanner probes from overwhelming error logging pipelines, database tables, and disk I/O.\n\n1. Never write 4xx errors (client errors, bad inputs, missing routes, scanner 404s) to the database or persistent logging storage.\n2. Only write 5xx errors (unhandled exceptions, database disconnects, crashes) to database error tables.\n3. Categorize console/stdout log levels:\n  - `404 Not Found` : Log at`debug` level only. In production environments where debug logs are silenced, scanner noise generates zero log volume.\n  - Other `4xx` (400, 401, 403, 422): Log at`warn` level.\n  - `5xx` : Log at`error` level with full stack traces.\n4. Ensure response bodies never expose internal database column names, raw queries, or stack traces to clients.\n\nPrevent automated brute-forcing of numeric OTP codes across clustered backend instances.\n\n1. Track verification attempts per recipient identifier in Redis: `auth:otp_attempts:<normalized_identifier>` .\n2. Normalize identifier (e.g. lowercase and trim email/phone).\n3. On verification attempt:\n  - Increment attempts: `INCR auth:otp_attempts:<identifier>` .\n  - On the first attempt, set TTL equal to the OTP expiration duration (e.g. 5 minutes / 300 seconds).\n4. Increment attempts: \n5. Enforce strict attempt threshold (maximum 5 attempts):\n  - If `attempts > 5` :\n    - Delete all pending OTP records for this recipient in the database immediately (hard revocation).\n    - Remove the attempt key from Redis.\n    - Throw a generic client exception: `\"Too many invalid attempts. Your verification code has expired. Please request a new code.\"`\n6. If \n7. Wrap Redis logic with fail-open handling: if Redis fails, proceed to database OTP verification so users are not locked out due to cache outages.\n\n``` js\nimport { NestFactory } from '@nestjs/core';\nimport { NestExpressApplication } from '@nestjs/platform-express';\nimport { AppModule } from './app.module';\n\nconst TRUSTED_PROXIES = [\n  'loopback',\n  'linklocal',\n  'uniquelocal',\n  // Cloudflare IPv4\n  '173.245.48.0/20',\n  '103.21.244.0/22',\n  '103.22.200.0/22',\n  '103.31.4.0/22',\n  '141.101.64.0/18',\n  '108.162.192.0/18',\n  '190.93.240.0/20',\n  '188.114.96.0/20',\n  '197.234.240.0/22',\n  '198.41.128.0/17',\n  '162.158.0.0/15',\n  '104.16.0.0/13',\n  '104.24.0.0/14',\n  '172.64.0.0/13',\n  '131.0.72.0/22',\n  // Cloudflare IPv6\n  '2400:cb00::/32',\n  '2606:4700::/32',\n  '2803:f800::/32',\n  '2405:b500::/32',\n  '2405:8100::/32',\n  '2a06:98c0::/29',\n  '2c0f:f248::/32',\n];\n\nasync function bootstrap() {\n  const app = await NestFactory.create<NestExpressApplication>(AppModule);\n\n  // Set trust proxy BEFORE any middleware reads req.ip\n  app.set('trust proxy', TRUSTED_PROXIES);\n\n  const port = process.env.PORT || 3000;\n  await app.listen(port);\n}\nbootstrap();\njs\nimport { Injectable, OnModuleDestroy, OnModuleInit, Logger, InternalServerErrorException } from '@nestjs/common';\nimport Redis from 'ioredis';\n\n@Injectable()\nexport class RedisService implements OnModuleInit, OnModuleDestroy {\n  private readonly logger = new Logger(RedisService.name);\n  private _isReady = false;\n  public readonly client: Redis;\n\n  constructor() {\n    const redisUrl = process.env.REDIS_URL;\n    if (!redisUrl) {\n      throw new InternalServerErrorException('REDIS_URL environment variable is missing');\n    }\n\n    this.client = new Redis(redisUrl, {\n      commandTimeout: Number(process.env.REDIS_COMMAND_TIMEOUT) || 80,\n      enableOfflineQueue: false,\n      maxRetriesPerRequest: 1,\n      retryStrategy: (times) => Math.min(times * 100, 3000),\n    });\n\n    this.client.on('connect', () => {\n      this._isReady = true;\n      this.logger.log('Redis connected');\n    });\n\n    this.client.on('ready', () => {\n      this._isReady = true;\n    });\n\n    this.client.on('error', (err: Error) => {\n      this._isReady = false;\n      this.logger.warn(`Redis connection error: ${err.message}`);\n    });\n\n    this.client.on('close', () => {\n      this._isReady = false;\n    });\n  }\n\n  async onModuleInit() {\n    await new Promise<void>((resolve) => {\n      if (this._isReady) return resolve();\n      const timeout = setTimeout(() => {\n        if (!this._isReady) {\n          this.logger.warn('Redis not ready after 2s; running in degraded fail-open mode');\n        }\n        resolve();\n      }, 2000);\n\n      this.client.once('ready', () => {\n        clearTimeout(timeout);\n        resolve();\n      });\n    });\n  }\n\n  get isReady(): boolean {\n    return this._isReady;\n  }\n\n  async onModuleDestroy() {\n    await this.client.quit();\n  }\n}\njs\nimport { Injectable, Logger, NestMiddleware } from '@nestjs/common';\nimport { Request, Response, NextFunction } from 'express';\nimport { RedisService } from '../redis/redis.service';\n\nconst SCANNER_REGEX =\n  /(?:^\\/|\\/)(?:\\.env[^/]*|\\.git[^/]*|\\.svn[^/]*|\\.aws[^/]*|\\.ssh[^/]*|\\.npmrc[^/]*|\\.netrc[^/]*|\\.DS_Store[^/]*|[^/]*_history|wp-admin|wp-login|wp-content|wp-includes|wp-config|phpmyadmin|pma|xmlrpc\\.php|cgi-bin|actuator|eval-stdin\\.php|web\\.config)(?:\\/|$|\\.|\\?)|(?:\\.php|\\.sql|\\.bak|\\.old|\\.orig|\\.save|\\.swp|\\.zip|\\.tar|\\.gz|\\.tgz|\\.bz2|\\.rar|\\.7z)(?:\\?.*)?$/i;\n\nconst SAFE_ROUTE_PREFIXES = [\n  '/api/v1',\n  '/api/docs',\n  '/health',\n  '/favicon.ico',\n];\n\nconst BAN_DURATIONS_SEC = [0, 900, 1800, 3600];\nconst STRIKE_WINDOW_SEC = Number(process.env.SCANNER_STRIKE_WINDOW_SEC) || 60;\nconst MAX_STRIKES = Number(process.env.SCANNER_MAX_STRIKES) || 3;\n\n@Injectable()\nexport class ScannerBlockMiddleware implements NestMiddleware {\n  private readonly logger = new Logger(ScannerBlockMiddleware.name);\n  private lastRedisWarnMs = 0;\n  private readonly REDIS_WARN_COOLDOWN_MS = 30_000;\n\n  constructor(private readonly redisService: RedisService) {}\n\n  async use(req: Request, res: Response, next: NextFunction) {\n    const clientIp = req.ip;\n    if (!clientIp) return next();\n\n    // 1. Intercept active bans\n    if (this.redisService.isReady) {\n      try {\n        const isBanned = await this.redisService.client.exists(`scan:ban:${clientIp}`);\n        if (isBanned) {\n          return res.status(403).send('Forbidden');\n        }\n      } catch (err: any) {\n        this.warnRedisOnce(err.message);\n      }\n    }\n\n    // 2. Decode path and check for malformed URIs\n    const rawPath = req.baseUrl ? `${req.baseUrl}${req.path}` : req.path;\n    let decodedPath = rawPath;\n    let isMalformed = false;\n    try {\n      decodedPath = decodeURIComponent(rawPath);\n    } catch {\n      isMalformed = true;\n    }\n\n    // 3. Whitelist bypass for legitimate API traffic\n    if (!isMalformed && SAFE_ROUTE_PREFIXES.some((p) => decodedPath.startsWith(p))) {\n      return next();\n    }\n\n    // 4. Scanner pattern check\n    if (!isMalformed && !SCANNER_REGEX.test(decodedPath)) {\n      return next();\n    }\n\n    // 5. Strike recording and progressive banning\n    if (this.redisService.isReady) {\n      try {\n        await this.recordStrike(clientIp, req.method, decodedPath);\n      } catch (err: any) {\n        this.warnRedisOnce(err.message);\n      }\n    }\n\n    // Return generic 404 to avoid revealing the honeypot\n    return res.status(404).end();\n  }\n\n  private async recordStrike(ip: string, method: string, path: string): Promise<void> {\n    const strikeKey = `scan:strike:${ip}`;\n    const banKey = `scan:ban:${ip}`;\n    const offenseKey = `scan:offenses:${ip}`;\n\n    const strikes = await this.redisService.client.incr(strikeKey);\n    if (strikes === 1) {\n      await this.redisService.client.expire(strikeKey, STRIKE_WINDOW_SEC);\n    }\n\n    if (strikes < MAX_STRIKES) {\n      this.logger.debug(`Scanner probe detected from ${ip}`);\n      return;\n    }\n\n    const offenses = await this.redisService.client.incr(offenseKey);\n    if (offenses === 1) {\n      await this.redisService.client.expire(offenseKey, 86400);\n    }\n\n    const banSec = BAN_DURATIONS_SEC[Math.min(offenses, BAN_DURATIONS_SEC.length - 1)];\n    await this.redisService.client.set(banKey, '1', 'EX', banSec);\n    await this.redisService.client.del(strikeKey);\n\n    this.logger.warn(`Temporarily blocked suspicious IP for ${banSec / 60}m`);\n  }\n\n  private warnRedisOnce(message: string): void {\n    const now = Date.now();\n    if (now - this.lastRedisWarnMs > this.REDIS_WARN_COOLDOWN_MS) {\n      this.lastRedisWarnMs = now;\n      this.logger.warn(`ScannerBlock: Redis unavailable (${message}). Failing open - checks skipped.`);\n    }\n  }\n}\njs\nimport {\n  Catch,\n  ExceptionFilter,\n  ExecutionContext,\n  HttpException,\n  HttpStatus,\n  Logger,\n  Optional,\n} from '@nestjs/common';\nimport { Request, Response } from 'express';\n\nexport interface PersistentErrorLogger {\n  logError(entry: any): Promise<void>;\n}\n\n@Catch()\nexport class GlobalExceptionFilter implements ExceptionFilter {\n  private readonly logger = new Logger(GlobalExceptionFilter.name);\n\n  constructor(@Optional() private errorLoggerService?: PersistentErrorLogger) {}\n\n  async catch(exception: unknown, host: ExecutionContext) {\n    const ctx = host.switchToHttp();\n    const request = ctx.getRequest<Request>();\n    const response = ctx.getResponse<Response>();\n\n    let status = HttpStatus.INTERNAL_SERVER_ERROR;\n    let message = 'Internal server error';\n    let stack: string | undefined;\n\n    if (exception instanceof HttpException) {\n      status = exception.getStatus();\n      const exceptionResponse = exception.getResponse();\n      message =\n        typeof exceptionResponse === 'object' && 'message' in exceptionResponse\n          ? (exceptionResponse as any).message\n          : exception.message;\n    } else if (exception instanceof Error) {\n      message = exception.message;\n      stack = exception.stack;\n    }\n\n    const is4xx = status >= 400 && status < 500;\n\n    // Never write 4xx errors to database to prevent storage bloat\n    if (!is4xx && this.errorLoggerService) {\n      try {\n        await this.errorLoggerService.logError({\n          message,\n          stack,\n          statusCode: status,\n          endpoint: request.url,\n          method: request.method,\n          context: 'GlobalExceptionFilter',\n          metadata: {\n            path: request.path,\n            ip: request.ip,\n            userAgent: request.get('user-agent'),\n          },\n        });\n      } catch (logError) {\n        this.logger.error('Failed to persist error log', logError);\n      }\n    }\n\n    // Scale logging by severity\n    if (status === HttpStatus.NOT_FOUND) {\n      this.logger.debug(`${request.method} ${request.url} - 404`);\n    } else if (is4xx) {\n      this.logger.warn(`${request.method} ${request.url} - ${status}`);\n    } else {\n      this.logger.error(`${request.method} ${request.url} - ${status}`, stack);\n    }\n\n    // Sanitize 500 response so internal traces are not exposed to clients\n    const clientMessage =\n      status >= 500 ? 'Internal server error' : message;\n\n    response.status(status).json({\n      statusCode: status,\n      message: clientMessage,\n      timestamp: new Date().toISOString(),\n      path: request.url,\n    });\n  }\n}\njs\nimport { Injectable, BadRequestException } from '@nestjs/common';\nimport { RedisService } from '../redis/redis.service';\n\n@Injectable()\nexport class AuthService {\n  private readonly otpValiditySeconds = 300; // 5 minutes\n  private readonly maxOtpAttempts = 5;\n\n  constructor(\n    private readonly redisService: RedisService,\n    private readonly usersService: any, // Injected user/persistence service\n  ) {}\n\n  async verifyOtp(otp: string, identifier: string): Promise<boolean> {\n    const cleanIdentifier = identifier.trim().toLowerCase();\n    const attemptKey = `auth:otp_attempts:${cleanIdentifier}`;\n\n    if (this.redisService.isReady) {\n      try {\n        const attempts = await this.redisService.client.incr(attemptKey);\n\n        if (attempts === 1) {\n          await this.redisService.client.expire(attemptKey, this.otpValiditySeconds);\n        }\n\n        if (attempts > this.maxOtpAttempts) {\n          // Immediately revoke active OTPs in the database\n          const user = await this.usersService.findOneByIdentifier(cleanIdentifier);\n          if (user) {\n            await this.usersService.deleteAllRelatedOtps(user.id);\n          }\n          await this.redisService.client.del(attemptKey);\n\n          throw new BadRequestException(\n            'Too many invalid attempts. Your verification code has expired. Please request a new code.',\n          );\n        }\n      } catch (err: any) {\n        if (err instanceof BadRequestException) throw err;\n        // Fail-open: proceed with database check if Redis is unavailable\n      }\n    }\n\n    // Proceed with validating the OTP code against database record...\n    return true;\n  }\n}\n```\n\nIf implementing in another language or backend framework, adhere to these structural equivalents:\n\n| Responsibility | Fastify (Node.js) | FastAPI / Starlette (Python) | Gin / Fiber (Go) | \n|---|---|---|---|\n| **Proxy Trust** | `fastify({ trustProxy: TRUSTED_PROXIES })` | Uvicorn `--forwarded-allow-ips` or Starlette`ProxyHeadersMiddleware` | `router.SetTrustedProxies(TRUSTED_PROXIES)` | \n| **Scanner Interceptor** | `addHook('onRequest', ...)` | Custom ASGI middleware `BaseHTTPMiddleware` | Standard middleware handler `c.Next()` | \n| **Redis Commands** | `ioredis` with 80ms command timeout | `redis-py` (`asyncio` ) with`socket_timeout=0.08` | `go-redis` with`DialTimeout: 80ms` | \n| **Error Handling** | `setErrorHandler` filtering 4xx from DB | Exception handler `app.add_exception_handler` | Global recovery middleware | \n| **Fail-Open Catch** | Wrap Redis calls in `try/except` -> fallback to`next()` | Wrap Redis calls in `try/except` -> proceed with request | Check `err != nil` -> proceed without blocking | \n\nWhen the implementation is complete, verify behavior with these tests:\n\n1. \n**Proxy Trust Resolution:**\n\n```\ncurl -H \"X-Forwarded-For: 203.0.113.195, 172.64.0.1\" http://localhost:3000/health\n```\n\n Verify the application resolves `req.ip` as`203.0.113.195` (client IP) and not Cloudflare's IP.\n2. \n**Scanner Detection & Stealth 404:**\n\n```\ncurl -i http://localhost:3000/.env\ncurl -i http://localhost:3000/wp-login.php\n```\n\n Verify the response is `404 Not Found` (empty body or standard 404, never 403 or custom scanner message).\n3. \n**Strike Escalation and Ban Activation:** Send probe requests exceeding the configured threshold within the sliding window:\n\n```\nfor i in {1..3}; do curl -s -o /dev/null -w \"%{http_code}\\n\" http://localhost:3000/.env; done\n```\n\n Subsequent request to any route: \n\n```\ncurl -i http://localhost:3000/api/v1/health\n```\n\n Verify response is HTTP `403 Forbidden` with a generic response body.\n4. \n**Whitelist Verification:**\n\n```\ncurl -i http://localhost:3000/api/v1/health\n```\n\n Verify legitimate routes under `/api/v1` are never intercepted by the scanner regex.\n5. \n**Fail-Open Resilience Test:** Stop the Redis instance or block its port.\nExecute requests to both legitimate and probed endpoints.\nVerify the application does not hang, throw 500 errors, or exceed 80ms latency.", "url": "https://wpnews.pro/news/prompt-for-model-to-add-protection-for-your-backend-from-scanners", "canonical_source": "https://gist.github.com/joelwillSeek/6ae65ec0c1a947814b392a9cee6498b9", "published_at": "2026-10-16 08:52:15+00:00", "updated_at": "2026-10-21 02:18:12.481135+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "developer-tools"], "entities": ["Shodan", "Censys", "Cloudflare", "Nginx", "Redis"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/prompt-for-model-to-add-protection-for-your-backend-from-scanners", "markdown": "https://wpnews.pro/news/prompt-for-model-to-add-protection-for-your-backend-from-scanners.md", "text": "https://wpnews.pro/news/prompt-for-model-to-add-protection-for-your-backend-from-scanners.txt", "jsonld": "https://wpnews.pro/news/prompt-for-model-to-add-protection-for-your-backend-from-scanners.jsonld"}}