Bulwark Gateway – fail-closed security proxy for LLM agents (self-hosted)

Bulwark Gateway, a self-hosted fail-closed security proxy for LLM agents, intercepts and validates tool calls between users and LLM backends, blocking threats immediately with 400+ detection patterns and 4 threat intel feeds. The proxy adds less than 40ms overhead and supports multi-tenancy, streaming tool call buffering, and SIEM integration across 13 platforms.

Security guardrail proxy for AI agents in cloud environments. Intercepts, validates, and enforces policies on tool calls between users and LLM agents. Designed for environments where the user is potentially adversarial fail-closed by default . Overview overview Architecture architecture Features features How It Compares how-it-compares Quick Start quick-start Configuration Summary configuration-summary Admin Portal admin-portal Documentation documentation Project Structure project-structure Development development License license Bulwark Gateway sits between your users/applications and your LLM backends OpenAI, Ollama, vLLM, Azure, etc. . Every request passes through multiple security layers before reaching the backend: Authentication — JWT/API key validation fail-closed Input Guardrail — Detects prompt injections, jailbreaks, encoding evasion IOC Check — Scans for malicious URLs/IPs/domains from threat intel feeds Tool Policy — RBAC enforcement per tenant/agent Output Filter — Redacts secrets/PII, detects indirect injection in responses Rate Limiter — Per-tenant request throttling via Redis If any layer detects a threat, the request is blocked immediately fail-closed . ┌──────────────────────────────────────────────┐ │ Bulwark Gateway │ │ │ User Request ─────► Auth ► Input Guardrail ► IOC Check │ X-Tenant-ID │ │ │ X-Agent-ID │ Agent Registry │ │ multi-backend │ │ │ │ │ Forward to backend │ │ │ │ │ Tool Policy ◄── Response ──► Output Filter │ └──────────────┼──────────────────────────────┘ │ ┌─────────────────────────┼─────────────────────────┐ ▼ ▼ ▼ Backend A RAG Backend B LLM Backend C Agent | Component | Port | Description | |---|---|---| Proxy | 8080 | Security hot path — intercepts all LLM requests | Admin Portal | 8090 | Web UI for configuration, monitoring, audit logs | Redis | 6379 | Rate limiting, state, session management | Prometheus | 9090 | Metrics collection | Grafana | 3000 | Dashboards and visualization | Full architecture details: docs/ARCHITECTURE.md Multi-tenant, multi-agent — Route requests to different backends per tenant/agent Zero-LLM hot path — Only regex + Pydantic + cache; p95 < 40ms overhead 400+ detection patterns — Prompt injection, jailbreak, encoding evasion, multilingual ES/ZH/AR 4 threat intel feeds — URLhaus, ThreatFox, AlienVault OTX, AbuseIPDB + MISP, OpenCTI, VirusTotal, Shodan Streaming tool call buffering — Tool calls validated BEFORE yielding to client Self-protection — Blocks agents from modifying gateway config Hot-reloadable — Policies, IOCs, and agent registry reload without restart Admin Portal — Full web UI for managing all aspects of the gateway SIEM integration — Export to 13 platforms Wazuh, Splunk, Elastic, QRadar, Datadog, etc. Notification channels — Slack, Teams, Discord, PagerDuty, Opsgenie, Telegram, Email, Google Chat Kubernetes-native — Full K8s manifests with NetworkPolicies, HPA, PDB, Pod Security Audit trail — Immutable log of all administrative changes Enterprise secrets — Vault, AWS SM, Azure KV, GCP SM, CyberArk, SealedSecrets Most LLM-security tools ship as a library/SDK you embed in your app code, or as a hosted SaaS you send your prompts to. Bulwark Gateway is a self-hosted, fail-closed proxy that sits in front of any OpenAI-compatible backend — no code changes in your app, no prompts leaving your network. | Capability | Bulwark Gateway | LLM-security SDKs LLM Guard, Guardrails AI, NeMo, Rebuff | Hosted SaaS Lakera, Prompt Security, etc. | |---|---|---|---| Deployment | Self-hosted proxy | Library in your app | Vendor cloud API call | Data leaves your network | No | No | Yes prompts sent to vendor | Code changes required | None drop-in proxy | Yes wrap every call | Yes SDK/API | Deterministic hot path | Yes — regex only, no LLM | Varies some call LLMs | Vendor-side opaque | Added latency | p95 < 40 ms in-cluster | Varies | Network round-trip to vendor | Multi-tenant / multi-agent routing | Built-in | No | Vendor-dependent | Tool-call / MCP RBAC | Yes per-agent policies | Rare | Vendor-dependent | Secret / PII output redaction | Yes | Some | Yes | SIEM export ECS / Wazuh / Splunk / … | Yes 13 platforms | No | Limited / vendor dashboard | Standalone scan API /v2/scan | Yes | N/A is the library | Yes is the API | Honest scope. Bulwark is a guardrail proxy , not a WAF and not a model-hosting platform. Classic SQLi/XSS on free-form chat input is not reliably matched by the input layer by design — those are enforced at the tool-argument layer where the payload actually reaches a DB/filesystem. See the published gap report /red-orbita/bulwark-gateway/blob/master/reports/blog-evidence/GAP-REPORT.md for exactly what it does and does not catch. The hot path is pure regex ~446 input + ~150 output patterns , so detection is fast and auditable but not a substitute for a semantic classifier on every edge case — ML scanners are available as an optional layer. Comparison reflects the common deployment model of each category; individual tools vary. Verify against each vendor's current capabilities. - Python 3.11+ - Docker 24+ - Kubernetes 1.28+ production or Docker Compose development - Redis 7+ Helm is the recommended path for managed clusters AKS/EKS/GKE . Build and push the images to your registry, then point the chart at them. <REGISTRY is your container registry path, e.g. myacr.azurecr.io , 123456789012.dkr.ecr.eu-west-1.amazonaws.com , or ghcr.io/my-org . 1. Build and push images to your registry run docker login <REGISTRY first docker build -t <REGISTRY /bulwark-gateway-proxy:1.0.0 -f Dockerfile . docker build -t <REGISTRY /bulwark-gateway-admin:1.0.0 -f docker/Dockerfile.admin . docker push <REGISTRY /bulwark-gateway-proxy:1.0.0 docker push <REGISTRY /bulwark-gateway-admin:1.0.0 2. Private registry only create the pull secret the pods use kubectl create namespace bulwark-gateway kubectl create secret docker-registry bulwark-registry \ --docker-server=<REGISTRY \ --docker-username=<USERNAME \ --docker-password=<PASSWORD \ -n bulwark-gateway 3. Install app secrets are auto-generated by the chart helm install bulwark ./helm/bulwark-gateway \ --namespace bulwark-gateway --create-namespace \ --set backend.ip=<YOUR LLM BACKEND IP \ --set proxy.image.repository=<REGISTRY /bulwark-gateway-proxy \ --set admin.image.repository=<REGISTRY /bulwark-gateway-admin \ --set proxy.image.tag=1.0.0 \ --set admin.image.tag=1.0.0 \ --set 'imagePullSecrets 0 .name=bulwark-registry' omit for public registries 4. Verify kubectl get pods -n bulwark-gateway helm test bulwark -n bulwark-gateway See docs/DEPLOYMENT.md /red-orbita/bulwark-gateway/blob/master/docs/DEPLOYMENT.md for external Redis, TLS/ingress, and full values.yaml reference. For local clusters, k8s/deploy.sh builds images, loads them into the cluster, and applies the Kustomize manifests in one step: 1. Generate secrets ./secrets/init.sh 2. Build + load + deploy auto-detects minikube/kind, generates secrets ./k8s/deploy.sh For a remote registry instead of local load: IMAGE REGISTRY=<REGISTRY / ./k8s/deploy.sh --backend-ip <IP 3. Verify kubectl get pods -n bulwark-gateway 1. Generate secrets ./secrets/init.sh 2. Start all services docker compose up -d 3. Access Proxy: http://localhost:8080 Admin: http://localhost:8090 Grafana: http://localhost:3000 Port-forward K8s kubectl port-forward svc/proxy 8080:8080 -n bulwark-gateway kubectl port-forward svc/admin 8090:8090 -n bulwark-gateway Or via Ingress: Proxy: https://bulwark-gateway.local Admin: https://admin.bulwark-gateway.local Health check curl http://localhost:8080/health Send a request API keys are passed as a Bearer token curl -X POST http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer your-api-key" \ -H "X-Tenant-ID: default" \ -H "X-Agent-ID: support-bot" \ -d '{"model": "gpt-4", "messages": {"role": "user", "content": "Hello"} }' Full deployment guide: docs/DEPLOYMENT.md | File | Purpose | |---|---| config/agents.yaml | Tenant → backend mapping, auth config | config/policies/ .yaml | Per-tenant security policies RBAC | config/notifications.yaml | Notification channel definitions | config/siem/ .yaml | SIEM platform templates | config/iocs.json | IOC database auto-updated by feeds | | Variable | Description | |---|---| BULWARK JWT SECRET | JWT signing key or FILE variant | BULWARK REDIS URL | Redis connection URL | BULWARK REDIS PASSWORD | Redis auth or FILE variant | BULWARK API KEYS | Comma-separated API keys or FILE | BULWARK WEBHOOK ALERT URLS | Legacy notification webhooks | BULWARK LOG LEVEL | Logging level INFO, DEBUG, etc. | All secrets support the FILE pattern — point an env var to a mounted file: env: - name: BULWARK JWT SECRET FILE value: /run/secrets/jwt-secret config/agents.yaml tenants: example-corp: backend url: "${BULWARK BACKEND URL:-http://ollama:11434}" auth token: "${BACKEND AUTH TOKEN}" allowed models: "gpt-4", "gpt-3.5-turbo" rate limit rpm: 60 config/policies/example-corp.yaml tenant id: example-corp tools: allowed: - web search - code interpreter blocked: - file system - shell exec guardrails: max tokens: 4096 block on injection: true Web-based management interface at / port 8090 . | Page | Function | |---|---| | Dashboard | Real-time metrics, recent blocks, sparklines | | Policies | CRUD, validation, hot-reload | | Guardrails | Pattern management, sandbox testing | | SIEM Export | Transport configuration, connectivity testing | Notifications | Alert channel management Slack, Teams, Email, etc. | | Audit Log | Immutable action history, export | | Orchestrator | Automated security testing | | Coverage Matrix | OWASP LLM Top 10 detection map | | IOCs | Threat intel feed management | | Tenants | Tenant registration and config | | Agents | Backend health monitoring | | Access Control | RBAC roles and permissions | | Enrichment | Attack replay browser, evasion telemetry, regex-candidate review | | Skills | Pre-deployment skill/MCP security scanner SkillSpector | | Plugins | Plugin hub — install, enable, security audit | | Evaluation | Red-team adversarial evaluation runner | | Discovery | Agent / shadow-AI / MCP discovery and risk assessment | | Status | System health Redis, proxy, scanner, telemetry | | User | Role | Default Password | Secret Key | |---|---|---|---| admin | Admin | bulwark-admin | ADMIN PASSWORD | security | Security | bulwark-security | SECURITY PASSWORD | auditor | Auditor | bulwark-auditor | AUDITOR PASSWORD | Change these immediately in production via K8s secrets. Detailed guides are in the docs/ /red-orbita/bulwark-gateway/blob/master/docs directory: | Document | Description | |---|---| INDEX | Architecture Deployment Operations Troubleshooting Notifications Security Hardening API Reference bulwark-gateway/ ├── src/ Proxy source code │ ├── main.py FastAPI app entry point │ ├── models.py Core data models SecurityEvent, Verdict │ ├── guardrails/ Detection engines │ │ ├── input guardrail.py Input analysis 400+ patterns │ │ ├── output filter.py Output redaction │ │ └── tool policy.py RBAC enforcement │ ├── routes/ API routes │ │ ├── proxy.py Main proxy flow hot path │ │ └── health.py Health/metrics endpoints │ ├── telemetry/ SIEM export + notifications │ │ ├── exporter.py Background batch exporter │ │ ├── notifications.py Multi-channel alert engine │ │ ├── queue.py Non-blocking event queue │ │ └── transports/ SIEM output adapters │ └── services/ Shared services IOC, registry ├── admin/ Admin portal │ ├── main.py Admin FastAPI app │ ├── routes/ Admin API routes │ ├── services/ Auth, audit, user store │ └── templates/ Jinja2 HTML templates UI ├── config/ Configuration │ ├── agents.yaml Agent/tenant registry │ ├── policies/ Security policy YAML files │ ├── notifications.yaml Notification channels │ └── siem/ SIEM platform templates ├── docs/ Detailed documentation ├── k8s/ Kubernetes manifests │ ├── base/ Core resources deployments, services │ ├── secrets/ Secret generation scripts │ └── monitoring/ Prometheus + Grafana ├── tests/ Test suite pytest, 1290+ tests ├── Dockerfile Proxy image ├── docker-compose.yml Development environment └── pyproject.toml Python project metadata Setup python -m venv .venv && source .venv/bin/activate pip install -e ". dev " Run server python -m uvicorn src.main:app --reload --port 8080 Run tests pytest -v Lint ruff check src/ tests/ Type check mypy src/ GPL-3.0-or-later — free to self-host, study, modify, and redistribute. See LICENSE /red-orbita/bulwark-gateway/blob/master/LICENSE . Contributions are welcome under the Contributor License Agreement /red-orbita/bulwark-gateway/blob/master/CLA.md , which lets you keep your copyright while keeping the project sustainably licensed. Licensing details and options: LICENSING.md .