AI Firewall – Security Gateway and Reverse Proxy for LLM Traffic

AI Firewall, an open-source security gateway for LLM traffic, launched as an MVP with a TypeScript gateway and Go edge proxy that inspect prompts and outputs for secrets, prompt injections, and EU AI Act compliance, though its detection layer remains deterministic and structurally weak against paraphrased or multilingual attacks.

AI Firewall is a security gateway that sits between your applications and LLM APIs OpenAI, Anthropic, Gemini, Ollama, vLLM . It inspects prompts before they leave your perimeter and model outputs before they reach your users: DLP / secret redaction — AWS keys, API tokens, JWTs, private-key headers, SSNs, credit cards, emails, phone numbers are masked or blocked. Prompt-injection & jailbreak filtering — persona-override phrases, system prompt exfiltration attempts, markdown-exfiltration pixels, zero-width steganography. EU AI Act tagging — every request is categorized Art. 5 prohibited / Annex III high-risk / Art. 50 transparency / minimal risk and written to a persistent audit log. Streaming interception — the Go edge proxy redacts secrets in SSE streams, including secrets split across chunk boundaries verified by unit test . This is an honest MVP. The table below is the ground truth; nothing else is claimed. | Component | Status | |---|---| TypeScript gateway server.ts : inspection, multi-provider forwarding OpenAI / Anthropic / Gemini / Ollama / vLLM , file-backed persistence, hashed API-key auth on mutating endpoints, audit logs, metrics computed from real traffic | Working | Go edge proxy cmd/proxy : auth SHA-256 key hashes, constant-time compare , request inspection, genuine Aho-Corasick automaton for attack phrases + RE2 for secrets, SSE stream redaction with carry-window | Working — compiles, go vet clean, unit-tested | | Operations dashboard React 19 + Tailwind : playground, rules engine, stream visualizer, audit/SOC view, key manager, live benchmark, automated test suite | Working | Live benchmark screen: precision / recall / F1 / latency measured at runtime on a transparent 30-case labeled corpus including hard paraphrase negatives the regex engine is expected to miss | Working — no pre-recorded numbers anywhere | | ML detection stage ONNX transformer classifier for paraphrased / multilingual attacks | Not implemented — roadmap | | Distributed deployment, rate limiting, SIEM export, PostgreSQL storage | Not implemented — roadmap | Known limitation, stated plainly: the current detection layer is deterministic keyword automaton + regex . It is fast and precise on known patterns, and structurally weak against paraphrased or multilingual attacks. The benchmark screen makes this gap visible on purpose; closing it with an ONNX classifier e.g. a fine-tuned DeBERTa-v3 prompt-injection model via onnxruntime-node is the top roadmap item. npm install cp .env.example .env add provider keys, or leave FIREWALL DEMO MODE=true npm run dev http://localhost:3000 On first startup an admin key is generated and printed once only its SHA-256 hash is stored in data/api-keys.json . Paste it in the dashboard's Key Manager, or set FIREWALL ADMIN KEY in .env . Rule and key mutations require it; without provider credentials the proxy returns clearly-flagged simulated completions "simulated": true in demo mode, or HTTP 502 otherwise. go test ./... unit tests incl. split-secret stream redaction go build ./cmd/proxy UPSTREAM LLM URL="https://api.openai.com" \ FIREWALL CLIENT KEYS="your-client-key" \ PORT=8080 ./proxy POST /api/v1/proxy/chat/completions { "provider": "openai", "model": "gpt-4o-mini", "messages": { "role": "user", "content": "My AWS key is AKIAIOSFODNN7EXAMPLE. Summarize earnings." } } The prompt is sanitized before forwarding; the response includes a firewall block proxy latency, redaction count, EU AI Act category, audit log id and a simulated flag that is true only when no provider credentials are set. Other endpoints: POST /api/v1/inspect dry-run , GET/POST/PUT/DELETE /api/v1/rules mutations: admin , GET /api/v1/logs , GET /api/v1/metrics computed from real traffic only , GET/POST/DELETE /api/v1/auth/keys admin , POST /api/v1/testsuite/run , GET /api/v1/proxy/sse/stream labeled demo stream for the visualizer . CLIENT ── Go edge proxy :8080 Node gateway + dashboard :3000 ├─ SHA-256 key auth ├─ multi-provider forwarding ├─ Aho-Corasick keyword automaton ├─ inspection + audit persistence ├─ RE2 secret redaction ├─ hashed API-key management └─ SSE carry-window interceptor └─ live measured benchmarks │ UPSTREAM LLM API The two processes are independent: the Go proxy is the low-latency data-path component; the Node gateway is the management plane and demo surface. Wiring the dashboard's rule store to the Go proxy over HTTP is on the roadmap. - ONNX transformer classifier as a second detection stage, benchmarked on public prompt-injection datasets with reproducible scripts. - Rule synchronization between the Node management plane and the Go proxy. - Reproducible load-test harness k6 with published p50/p99 overhead. - PostgreSQL audit storage, retention policies, SIEM syslog/OTLP export. - Rate limiting per API key; mTLS between proxy and management plane. Business Source License 1.1 BUSL-1.1 — see the LICENSE /bossandboss/Ai-Firewall/blob/main/LICENSE file. In short: source-available; free for non-production use and for production use up to 10,000 inspected requests per day; offering the software as a competing hosted/managed service requires a commercial license. On the Change Date 2030-07-23 the code automatically converts to Apache 2.0. Before publishing: fill in the Licensor name and contact email in the LICENSE parameters, and have the final text reviewed by an IP lawyer.