# Open-source LLM Firewall with reversible PII scrubbing and FinOps routing

> Source: <https://github.com/67spp/sentinsentinel-ai-gateway>
> Published: 2026-11-04 18:30:45+00:00

**Enterprise-Grade LLM Firewall, Zero-Trust PII Scrubber & FinOps Routing Engine**

**Sentinel AI Gateway** is a lightweight, ultra-low-latency proxy designed for enterprises adopting Generative AI. It sits between your internal microservices/employees and external LLM providers (OpenAI, Anthropic, AWS Bedrock), solving the two biggest blockers in Enterprise AI: **Data Leakage (DLP)** and **Uncontrolled API Costs (FinOps)**.

``` php
graph LR
    A[Internal App / Employee] -->|Raw Prompt with Secrets| B(Sentinel Gateway)
    subgraph Zero-Trust Corporate Perimeter
    B --> C{PII & Secret Scrubber}
    C -->|Vaults PII Locally| D[(Encrypted In-Memory Vault)]
    C -->|Sanitized Prompt| E{Semantic Cache & FinOps Router}
    end
    E -->|Cache Hit: $0 Cost| B
    E -->|Low Complexity| F[Local Llama-3 8B]
    E -->|High Complexity| G[Cloud GPT-4o / Claude]
    F --> D
    G --> D
    D -->|Restores PII into Response| A
```

| Enterprise Challenge | Traditional Approach | Sentinel AI Gateway Approach | Business Impact | 
|---|---|---|---|
| **Data Privacy (GDPR, HIPAA, SOC2)** | Blocking AI completely or relying on legal NDAs | **Automated Reversible Tokenization** : Strips emails, CC numbers, SSNs, and API keys*before* leaving the network. | **0% PII leakage** to 3rd-party model providers. | 
| **Runaway API Bills** | Hard rate-limiting developers | **Semantic Caching & Smart Routing** : Routes simple tasks to cheap local models and caches repetitive queries. | **Up to 45% reduction** in monthly LLM inference spend. | 
| **Vendor Lock-in** | Rewriting SDKs across 50+ microservices | **Unified OpenAI-Compatible Endpoint** : Switch providers in one config file. | **10x faster** migration between AI vendors. | 

Deploy the gateway in under 30 seconds:

```
git clone https://github.com/67spp/sentinel-ai-gateway.git
cd sentinel-ai-gateway
docker build -t sentinel-gateway .
docker run -p 8000:8000 sentinel-gateway
```

Once running, explore the interactive Swagger UI documentation at `http://localhost:8000/docs`.

**1. Outbound Request (from your developer):**

```
curl -X POST "http://localhost:8000/v1/chat/completions" \
     -H "Content-Type: application/json" \
     -d '{
       "prompt": "Send invoice to ceo@enterprise.com using card 4532-1111-2222-3333",
       "task_complexity": "auto"
     }'
```

**2. What the External Cloud LLM Actually Sees:**

`"Send invoice to [EMAIL_REDACTED_1] using card [CREDIT_CARD_REDACTED_2]"`

**3. What Sentinel Returns to Your Application:**

```
{
  "response": "Processed securely by local/llama-3-8b-instruct: Confirmed action for Send invoice to ceo@enterprise.com using card 4532-1111-2222-3333",
  "sanitized_prompt": "Send invoice to [EMAIL_REDACTED_1] using card [CREDIT_CARD_REDACTED_2]",
  "pii_detected_and_masked": 2,
  "routed_model": "local/llama-3-8b-instruct",
  "cache_hit": false,
  "estimated_cost_saved_usd": 0.0045,
  "latency_ms": 1.42
}
```

- Sub-2ms Regex & Pattern DLP Engine
- Reversible In-Memory Token Vault
- Complexity-based FinOps Routing
- Microsoft Presidio NLP integration for context-aware NER scrubbing
- Redis Vector DB integration for cosine-similarity semantic caching
- Role-Based Access Control (RBAC) & Prometheus/Grafana metrics export

*Architected for high-compliance environments. For enterprise integration, custom deployment, or acquisition inquiries, reach out via [LinkedIn](https://github.com/67spp/sentinsentinel-ai-gateway/blob/main/@billlvic).*