{"slug": "ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic", "title": "AI Firewall – Security Gateway and Reverse Proxy for LLM Traffic", "summary": "AI Firewall, an open-source security gateway for LLM traffic, launched as an MVP with a TypeScript gateway and Go edge proxy that inspect prompts and outputs for secrets, prompt injections, and EU AI Act compliance, though its detection layer remains deterministic and structurally weak against paraphrased or multilingual attacks.", "body_md": "AI Firewall is a security gateway that sits between your applications and LLM APIs (OpenAI, Anthropic, Gemini, Ollama, vLLM). It inspects prompts before they leave your perimeter and model outputs before they reach your users:\n\n**DLP / secret redaction**— AWS keys, API tokens, JWTs, private-key headers, SSNs, credit cards, emails, phone numbers are masked or blocked.** Prompt-injection & jailbreak filtering**— persona-override phrases, system prompt exfiltration attempts, markdown-exfiltration pixels, zero-width steganography.** EU AI Act tagging**— every request is categorized (Art. 5 prohibited / Annex III high-risk / Art. 50 transparency / minimal risk) and written to a persistent audit log.**Streaming interception**— the Go edge proxy redacts secrets in SSE streams, including secrets split across chunk boundaries (verified by unit test).\n\nThis is an honest MVP. The table below is the ground truth; nothing else is claimed.\n\n| Component | Status |\n|---|---|\nTypeScript gateway (`server.ts` ): inspection, multi-provider forwarding (OpenAI / Anthropic / Gemini / Ollama / vLLM), file-backed persistence, hashed API-key auth on mutating endpoints, audit logs, metrics computed from real traffic |\nWorking |\nGo edge proxy (`cmd/proxy` ): auth (SHA-256 key hashes, constant-time compare), request inspection, genuine Aho-Corasick automaton for attack phrases + RE2 for secrets, SSE stream redaction with carry-window |\nWorking — compiles, `go vet` clean, unit-tested |\n| Operations dashboard (React 19 + Tailwind): playground, rules engine, stream visualizer, audit/SOC view, key manager, live benchmark, automated test suite | Working |\nLive benchmark screen: precision / recall / F1 / latency measured at runtime on a transparent 30-case labeled corpus (including hard paraphrase negatives the regex engine is expected to miss) |\nWorking — no pre-recorded numbers anywhere |\n| ML detection stage (ONNX transformer classifier for paraphrased / multilingual attacks) | Not implemented — roadmap |\n| Distributed deployment, rate limiting, SIEM export, PostgreSQL storage | Not implemented — roadmap |\n\nKnown limitation, stated plainly: the current detection layer is deterministic\n(keyword automaton + regex). It is fast and precise on known patterns, and\nstructurally weak against paraphrased or multilingual attacks. The benchmark\nscreen makes this gap visible on purpose; closing it with an ONNX classifier\n(e.g. a fine-tuned DeBERTa-v3 prompt-injection model via `onnxruntime-node`\n\n)\nis the top roadmap item.\n\n```\nnpm install\ncp .env.example .env      # add provider keys, or leave FIREWALL_DEMO_MODE=true\nnpm run dev               # http://localhost:3000\n```\n\nOn first startup an **admin key** is generated and printed once (only its\nSHA-256 hash is stored in `data/api-keys.json`\n\n). Paste it in the dashboard's\nKey Manager, or set `FIREWALL_ADMIN_KEY`\n\nin `.env`\n\n. Rule and key mutations\nrequire it; without provider credentials the proxy returns clearly-flagged\nsimulated completions (`\"simulated\": true`\n\n) in demo mode, or HTTP 502 otherwise.\n\n```\ngo test ./...             # unit tests incl. split-secret stream redaction\ngo build ./cmd/proxy\nUPSTREAM_LLM_URL=\"https://api.openai.com\" \\\nFIREWALL_CLIENT_KEYS=\"your-client-key\" \\\nPORT=8080 ./proxy\n```\n\n`POST /api/v1/proxy/chat/completions`\n\n```\n{\n  \"provider\": \"openai\",\n  \"model\": \"gpt-4o-mini\",\n  \"messages\": [{ \"role\": \"user\", \"content\": \"My AWS key is AKIAIOSFODNN7EXAMPLE. Summarize earnings.\" }]\n}\n```\n\nThe prompt is sanitized before forwarding; the response includes a `firewall`\n\nblock (proxy latency, redaction count, EU AI Act category, audit log id) and a\n`simulated`\n\nflag that is `true`\n\nonly when no provider credentials are set.\n\nOther endpoints: `POST /api/v1/inspect`\n\n(dry-run), `GET/POST/PUT/DELETE /api/v1/rules`\n\n(mutations: admin), `GET /api/v1/logs`\n\n, `GET /api/v1/metrics`\n\n(computed from real traffic only), `GET/POST/DELETE /api/v1/auth/keys`\n\n(admin),\n`POST /api/v1/testsuite/run`\n\n, `GET /api/v1/proxy/sse/stream`\n\n(labeled demo\nstream for the visualizer).\n\n```\n[ CLIENT ] ──> [ Go edge proxy :8080 ]              [ Node gateway + dashboard :3000 ]\n                  ├─ SHA-256 key auth                  ├─ multi-provider forwarding\n                  ├─ Aho-Corasick keyword automaton    ├─ inspection + audit persistence\n                  ├─ RE2 secret redaction              ├─ hashed API-key management\n                  └─ SSE carry-window interceptor      └─ live measured benchmarks\n                        │\n                 [ UPSTREAM LLM API ]\n```\n\nThe two processes are independent: the Go proxy is the low-latency data-path component; the Node gateway is the management plane and demo surface. Wiring the dashboard's rule store to the Go proxy over HTTP is on the roadmap.\n\n- ONNX transformer classifier as a second detection stage, benchmarked on public prompt-injection datasets with reproducible scripts.\n- Rule synchronization between the Node management plane and the Go proxy.\n- Reproducible load-test harness (\n`k6`\n\n) with published p50/p99 overhead. - PostgreSQL audit storage, retention policies, SIEM (syslog/OTLP) export.\n- Rate limiting per API key; mTLS between proxy and management plane.\n\nBusiness Source License 1.1 (BUSL-1.1) — see the [LICENSE](/bossandboss/Ai-Firewall/blob/main/LICENSE) file.\n\nIn short: source-available; free for non-production use and for production use up to 10,000 inspected requests per day; offering the software as a competing hosted/managed service requires a commercial license. On the Change Date (2030-07-23) the code automatically converts to Apache 2.0.\n\nBefore publishing: fill in the Licensor name and contact email in the LICENSE parameters, and have the final text reviewed by an IP lawyer.", "url": "https://wpnews.pro/news/ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic", "canonical_source": "https://github.com/bossandboss/Ai-Firewall", "published_at": "2026-07-23 07:47:13+00:00", "updated_at": "2026-07-23 07:52:18.048680+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy", "ai-tools", "ai-infrastructure"], "entities": ["AI Firewall", "OpenAI", "Anthropic", "Gemini", "Ollama", "vLLM", "ONNX", "DeBERTa-v3"], "alternates": {"html": "https://wpnews.pro/news/ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic", "markdown": "https://wpnews.pro/news/ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic.md", "text": "https://wpnews.pro/news/ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic.txt", "jsonld": "https://wpnews.pro/news/ai-firewall-security-gateway-and-reverse-proxy-for-llm-traffic.jsonld"}}