{"slug": "bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted", "title": "Bulwark Gateway – fail-closed security proxy for LLM agents (self-hosted)", "summary": "Bulwark Gateway, a self-hosted fail-closed security proxy for LLM agents, intercepts and validates tool calls between users and LLM backends, blocking threats immediately with 400+ detection patterns and 4 threat intel feeds. The proxy adds less than 40ms overhead and supports multi-tenancy, streaming tool call buffering, and SIEM integration across 13 platforms.", "body_md": "Security guardrail proxy for AI agents in cloud environments.\n\nIntercepts, validates, and enforces policies on tool calls between users and LLM agents. Designed for environments where **the user is potentially adversarial** (fail-closed by default).\n\n[Overview](#overview)[Architecture](#architecture)[Features](#features)[How It Compares](#how-it-compares)[Quick Start](#quick-start)[Configuration Summary](#configuration-summary)[Admin Portal](#admin-portal)[Documentation](#documentation)[Project Structure](#project-structure)[Development](#development)[License](#license)\n\nBulwark Gateway sits between your users/applications and your LLM backends (OpenAI, Ollama, vLLM, Azure, etc.). Every request passes through multiple security layers before reaching the backend:\n\n**Authentication**— JWT/API key validation (fail-closed)** Input Guardrail**— Detects prompt injections, jailbreaks, encoding evasion** IOC Check**— Scans for malicious URLs/IPs/domains from threat intel feeds** Tool Policy**— RBAC enforcement per tenant/agent** Output Filter**— Redacts secrets/PII, detects indirect injection in responses** Rate Limiter**— Per-tenant request throttling via Redis\n\nIf any layer detects a threat, the request is **blocked immediately** (fail-closed).\n\n```\n                    ┌──────────────────────────────────────────────┐\n                    │             Bulwark Gateway                  │\n                    │                                              │\n User Request ─────►  Auth ► Input Guardrail ► IOC Check          │\n  X-Tenant-ID      │                              │               │\n  X-Agent-ID       │                    Agent Registry             │\n                    │                    (multi-backend)            │\n                    │                         │                    │\n                    │              Forward to backend              │\n                    │                         │                    │\n                    │  Tool Policy ◄── Response ──► Output Filter  │\n                    └──────────────┼──────────────────────────────┘\n                                   │\n         ┌─────────────────────────┼─────────────────────────┐\n         ▼                         ▼                         ▼\n  Backend A (RAG)          Backend B (LLM)        Backend C (Agent)\n```\n\n| Component | Port | Description |\n|---|---|---|\nProxy |\n8080 | Security hot path — intercepts all LLM requests |\nAdmin Portal |\n8090 | Web UI for configuration, monitoring, audit logs |\nRedis |\n6379 | Rate limiting, state, session management |\nPrometheus |\n9090 | Metrics collection |\nGrafana |\n3000 | Dashboards and visualization |\n\nFull architecture details:\n\n[docs/ARCHITECTURE.md]\n\n**Multi-tenant, multi-agent**— Route requests to different backends per tenant/agent** Zero-LLM hot path**— Only regex + Pydantic + cache; p95 < 40ms overhead** 400+ detection patterns**— Prompt injection, jailbreak, encoding evasion, multilingual (ES/ZH/AR)** 4 threat intel feeds**— URLhaus, ThreatFox, AlienVault OTX, AbuseIPDB (+ MISP, OpenCTI, VirusTotal, Shodan)** Streaming tool call buffering**— Tool calls validated BEFORE yielding to client** Self-protection**— Blocks agents from modifying gateway config** Hot-reloadable**— Policies, IOCs, and agent registry reload without restart** Admin Portal**— Full web UI for managing all aspects of the gateway** SIEM integration**— Export to 13 platforms (Wazuh, Splunk, Elastic, QRadar, Datadog, etc.)** Notification channels**— Slack, Teams, Discord, PagerDuty, Opsgenie, Telegram, Email, Google Chat** Kubernetes-native**— Full K8s manifests with NetworkPolicies, HPA, PDB, Pod Security** Audit trail**— Immutable log of all administrative changes** Enterprise secrets**— Vault, AWS SM, Azure KV, GCP SM, CyberArk, SealedSecrets\n\nMost LLM-security tools ship as a **library/SDK** you embed in your app code, or\nas a **hosted SaaS** you send your prompts to. Bulwark Gateway is a\n**self-hosted, fail-closed proxy** that sits in front of any OpenAI-compatible\nbackend — no code changes in your app, no prompts leaving your network.\n\n| Capability | Bulwark Gateway | LLM-security SDKs (LLM Guard, Guardrails AI, NeMo, Rebuff) | Hosted SaaS (Lakera, Prompt Security, etc.) |\n|---|---|---|---|\nDeployment |\nSelf-hosted proxy | Library in your app | Vendor cloud (API call) |\nData leaves your network |\nNo | No | Yes (prompts sent to vendor) |\nCode changes required |\nNone (drop-in proxy) | Yes (wrap every call) | Yes (SDK/API) |\nDeterministic hot path |\nYes — regex only, no LLM | Varies (some call LLMs) | Vendor-side (opaque) |\nAdded latency |\np95 < 40 ms (in-cluster) | Varies | Network round-trip to vendor |\nMulti-tenant / multi-agent routing |\nBuilt-in | No | Vendor-dependent |\nTool-call / MCP RBAC |\nYes (per-agent policies) | Rare | Vendor-dependent |\nSecret / PII output redaction |\nYes | Some | Yes |\nSIEM export (ECS / Wazuh / Splunk / …) |\nYes (13 platforms) | No | Limited / vendor dashboard |\nStandalone scan API (`/v2/scan` ) |\nYes | N/A (is the library) | Yes (is the API) |\n\n**Honest scope.** Bulwark is a *guardrail proxy*, not a WAF and not a\nmodel-hosting platform. Classic SQLi/XSS on free-form chat input is **not**\nreliably matched by the input layer by design — those are enforced at the\ntool-argument layer where the payload actually reaches a DB/filesystem. See the\npublished [gap report](/red-orbita/bulwark-gateway/blob/master/reports/blog-evidence/GAP-REPORT.md) for exactly what it\ndoes and does not catch. The hot path is pure regex (~446 input + ~150 output\npatterns), so detection is fast and auditable but not a substitute for a semantic\nclassifier on every edge case — ML scanners are available as an optional layer.\n\nComparison reflects the common deployment model of each category; individual tools vary. Verify against each vendor's current capabilities.\n\n- Python 3.11+\n- Docker 24+\n- Kubernetes 1.28+ (production) or Docker Compose (development)\n- Redis 7+\n\nHelm is the recommended path for managed clusters (AKS/EKS/GKE). Build and push the images to your registry, then point the chart at them.\n\n`<REGISTRY>`\n\nis your container registry path, e.g. `myacr.azurecr.io`\n\n,\n`123456789012.dkr.ecr.eu-west-1.amazonaws.com`\n\n, or `ghcr.io/my-org`\n\n.\n\n```\n# 1. Build and push images to your registry (run `docker login <REGISTRY>` first)\ndocker build -t <REGISTRY>/bulwark-gateway-proxy:1.0.0 -f Dockerfile .\ndocker build -t <REGISTRY>/bulwark-gateway-admin:1.0.0 -f docker/Dockerfile.admin .\ndocker push <REGISTRY>/bulwark-gateway-proxy:1.0.0\ndocker push <REGISTRY>/bulwark-gateway-admin:1.0.0\n\n# 2. (Private registry only) create the pull secret the pods use\nkubectl create namespace bulwark-gateway\nkubectl create secret docker-registry bulwark-registry \\\n  --docker-server=<REGISTRY> \\\n  --docker-username=<USERNAME> \\\n  --docker-password=<PASSWORD> \\\n  -n bulwark-gateway\n\n# 3. Install (app secrets are auto-generated by the chart)\nhelm install bulwark ./helm/bulwark-gateway \\\n  --namespace bulwark-gateway --create-namespace \\\n  --set backend.ip=<YOUR_LLM_BACKEND_IP> \\\n  --set proxy.image.repository=<REGISTRY>/bulwark-gateway-proxy \\\n  --set admin.image.repository=<REGISTRY>/bulwark-gateway-admin \\\n  --set proxy.image.tag=1.0.0 \\\n  --set admin.image.tag=1.0.0 \\\n  --set 'imagePullSecrets[0].name=bulwark-registry'   # omit for public registries\n\n# 4. Verify\nkubectl get pods -n bulwark-gateway\nhelm test bulwark -n bulwark-gateway\n```\n\nSee [docs/DEPLOYMENT.md](/red-orbita/bulwark-gateway/blob/master/docs/DEPLOYMENT.md) for external Redis, TLS/ingress, and\nfull `values.yaml`\n\nreference.\n\nFor local clusters, `k8s/deploy.sh`\n\nbuilds images, loads them into the cluster,\nand applies the Kustomize manifests in one step:\n\n```\n# 1. Generate secrets\n./secrets/init.sh\n\n# 2. Build + load + deploy (auto-detects minikube/kind, generates secrets)\n./k8s/deploy.sh\n\n# For a remote registry instead of local load:\n#   IMAGE_REGISTRY=<REGISTRY>/ ./k8s/deploy.sh --backend-ip <IP>\n\n# 3. Verify\nkubectl get pods -n bulwark-gateway\n# 1. Generate secrets\n./secrets/init.sh\n\n# 2. Start all services\ndocker compose up -d\n\n# 3. Access\n#    Proxy:  http://localhost:8080\n#    Admin:  http://localhost:8090\n#    Grafana: http://localhost:3000\n# Port-forward (K8s)\nkubectl port-forward svc/proxy 8080:8080 -n bulwark-gateway\nkubectl port-forward svc/admin 8090:8090 -n bulwark-gateway\n\n# Or via Ingress:\n#   Proxy:  https://bulwark-gateway.local\n#   Admin:  https://admin.bulwark-gateway.local\n# Health check\ncurl http://localhost:8080/health\n\n# Send a request (API keys are passed as a Bearer token)\ncurl -X POST http://localhost:8080/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -H \"Authorization: Bearer your-api-key\" \\\n  -H \"X-Tenant-ID: default\" \\\n  -H \"X-Agent-ID: support-bot\" \\\n  -d '{\"model\": \"gpt-4\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]}'\n```\n\nFull deployment guide:\n\n[docs/DEPLOYMENT.md]\n\n| File | Purpose |\n|---|---|\n`config/agents.yaml` |\nTenant → backend mapping, auth config |\n`config/policies/*.yaml` |\nPer-tenant security policies (RBAC) |\n`config/notifications.yaml` |\nNotification channel definitions |\n`config/siem/*.yaml` |\nSIEM platform templates |\n`config/iocs.json` |\nIOC database (auto-updated by feeds) |\n\n| Variable | Description |\n|---|---|\n`BULWARK_JWT_SECRET` |\nJWT signing key (or `*_FILE` variant) |\n`BULWARK_REDIS_URL` |\nRedis connection URL |\n`BULWARK_REDIS_PASSWORD` |\nRedis auth (or `*_FILE` variant) |\n`BULWARK_API_KEYS` |\nComma-separated API keys (or `*_FILE` ) |\n`BULWARK_WEBHOOK_ALERT_URLS` |\nLegacy notification webhooks |\n`BULWARK_LOG_LEVEL` |\nLogging level (INFO, DEBUG, etc.) |\n\nAll secrets support the ** *_FILE pattern** — point an env var to a mounted file:\n\n```\nenv:\n  - name: BULWARK_JWT_SECRET_FILE\n    value: /run/secrets/jwt-secret\n# config/agents.yaml\ntenants:\n  example-corp:\n    backend_url: \"${BULWARK_BACKEND_URL:-http://ollama:11434}\"\n    auth_token: \"${BACKEND_AUTH_TOKEN}\"\n    allowed_models: [\"gpt-4\", \"gpt-3.5-turbo\"]\n    rate_limit_rpm: 60\n# config/policies/example-corp.yaml\ntenant_id: example-corp\ntools:\n  allowed:\n    - web_search\n    - code_interpreter\n  blocked:\n    - file_system\n    - shell_exec\nguardrails:\n  max_tokens: 4096\n  block_on_injection: true\n```\n\nWeb-based management interface at `/`\n\n(port 8090).\n\n| Page | Function |\n|---|---|\n| Dashboard | Real-time metrics, recent blocks, sparklines |\n| Policies | CRUD, validation, hot-reload |\n| Guardrails | Pattern management, sandbox testing |\n| SIEM Export | Transport configuration, connectivity testing |\nNotifications |\nAlert channel management (Slack, Teams, Email, etc.) |\n| Audit Log | Immutable action history, export |\n| Orchestrator | Automated security testing |\n| Coverage Matrix | OWASP LLM Top 10 detection map |\n| IOCs | Threat intel feed management |\n| Tenants | Tenant registration and config |\n| Agents | Backend health monitoring |\n| Access Control | RBAC roles and permissions |\n| Enrichment | Attack replay browser, evasion telemetry, regex-candidate review |\n| Skills | Pre-deployment skill/MCP security scanner (SkillSpector) |\n| Plugins | Plugin hub — install, enable, security audit |\n| Evaluation | Red-team adversarial evaluation runner |\n| Discovery | Agent / shadow-AI / MCP discovery and risk assessment |\n| Status | System health (Redis, proxy, scanner, telemetry) |\n\n| User | Role | Default Password | Secret Key |\n|---|---|---|---|\n`admin` |\nAdmin | `bulwark-admin` |\n`ADMIN_PASSWORD` |\n`security` |\nSecurity | `bulwark-security` |\n`SECURITY_PASSWORD` |\n`auditor` |\nAuditor | `bulwark-auditor` |\n`AUDITOR_PASSWORD` |\n\nChange these immediately in production via K8s secrets.\n\nDetailed guides are in the [ docs/](/red-orbita/bulwark-gateway/blob/master/docs) directory:\n\n| Document | Description |\n|---|---|\nINDEX |\n\n**Architecture****Deployment****Operations****Troubleshooting****Notifications****Security Hardening****API Reference**\n\n```\nbulwark-gateway/\n├── src/                        # Proxy source code\n│   ├── main.py                 # FastAPI app entry point\n│   ├── models.py               # Core data models (SecurityEvent, Verdict)\n│   ├── guardrails/             # Detection engines\n│   │   ├── input_guardrail.py  # Input analysis (400+ patterns)\n│   │   ├── output_filter.py    # Output redaction\n│   │   └── tool_policy.py      # RBAC enforcement\n│   ├── routes/                 # API routes\n│   │   ├── proxy.py            # Main proxy flow (hot path)\n│   │   └── health.py           # Health/metrics endpoints\n│   ├── telemetry/              # SIEM export + notifications\n│   │   ├── exporter.py         # Background batch exporter\n│   │   ├── notifications.py    # Multi-channel alert engine\n│   │   ├── queue.py            # Non-blocking event queue\n│   │   └── transports/         # SIEM output adapters\n│   └── services/               # Shared services (IOC, registry)\n├── admin/                      # Admin portal\n│   ├── main.py                 # Admin FastAPI app\n│   ├── routes/                 # Admin API routes\n│   ├── services/               # Auth, audit, user store\n│   └── templates/              # Jinja2 HTML templates (UI)\n├── config/                     # Configuration\n│   ├── agents.yaml             # Agent/tenant registry\n│   ├── policies/               # Security policy YAML files\n│   ├── notifications.yaml      # Notification channels\n│   └── siem/                   # SIEM platform templates\n├── docs/                       # Detailed documentation\n├── k8s/                        # Kubernetes manifests\n│   ├── base/                   # Core resources (deployments, services)\n│   ├── secrets/                # Secret generation scripts\n│   └── monitoring/             # Prometheus + Grafana\n├── tests/                      # Test suite (pytest, 1290+ tests)\n├── Dockerfile                  # Proxy image\n├── docker-compose.yml          # Development environment\n└── pyproject.toml              # Python project metadata\n# Setup\npython -m venv .venv && source .venv/bin/activate\npip install -e \".[dev]\"\n\n# Run server\npython -m uvicorn src.main:app --reload --port 8080\n\n# Run tests\npytest -v\n\n# Lint\nruff check src/ tests/\n\n# Type check\nmypy src/\n```\n\n**GPL-3.0-or-later** — free to self-host, study, modify, and redistribute. See\n[ LICENSE](/red-orbita/bulwark-gateway/blob/master/LICENSE).\n\nContributions are welcome under the\n[Contributor License Agreement](/red-orbita/bulwark-gateway/blob/master/CLA.md), which lets you keep your copyright\nwhile keeping the project sustainably licensed.\n\nLicensing details and options:\n\n`LICENSING.md`\n\n.", "url": "https://wpnews.pro/news/bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted", "canonical_source": "https://github.com/red-orbita/bulwark-gateway", "published_at": "2026-08-21 08:18:00+00:00", "updated_at": "2026-08-21 08:43:53.484528+00:00", "lang": "en", "topics": ["ai-safety", "ai-infrastructure", "ai-tools", "ai-agents"], "entities": ["Bulwark Gateway", "OpenAI", "Ollama", "vLLM", "Azure", "Redis", "Prometheus", "Grafana"], "alternates": {"html": "https://wpnews.pro/news/bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted", "markdown": "https://wpnews.pro/news/bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted.md", "text": "https://wpnews.pro/news/bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted.txt", "jsonld": "https://wpnews.pro/news/bulwark-gateway-fail-closed-security-proxy-for-llm-agents-self-hosted.jsonld"}}