Enterprise-Grade LLM Firewall, Zero-Trust PII Scrubber & FinOps Routing Engine
Sentinel AI Gateway is a lightweight, ultra-low-latency proxy designed for enterprises adopting Generative AI. It sits between your internal microservices/employees and external LLM providers (OpenAI, Anthropic, AWS Bedrock), solving the two biggest blockers in Enterprise AI: Data Leakage (DLP) and Uncontrolled API Costs (FinOps).
graph LR
A[Internal App / Employee] -->|Raw Prompt with Secrets| B(Sentinel Gateway)
subgraph Zero-Trust Corporate Perimeter
B --> C{PII & Secret Scrubber}
C -->|Vaults PII Locally| D[(Encrypted In-Memory Vault)]
C -->|Sanitized Prompt| E{Semantic Cache & FinOps Router}
end
E -->|Cache Hit: $0 Cost| B
E -->|Low Complexity| F[Local Llama-3 8B]
E -->|High Complexity| G[Cloud GPT-4o / Claude]
F --> D
G --> D
D -->|Restores PII into Response| A
| Enterprise Challenge | Traditional Approach | Sentinel AI Gateway Approach | Business Impact |
|---|---|---|---|
| Data Privacy (GDPR, HIPAA, SOC2) | Blocking AI completely or relying on legal NDAs | Automated Reversible Tokenization : Strips emails, CC numbers, SSNs, and API keysbefore leaving the network. | 0% PII leakage to 3rd-party model providers. |
| Runaway API Bills | Hard rate-limiting developers | Semantic Caching & Smart Routing : Routes simple tasks to cheap local models and caches repetitive queries. | Up to 45% reduction in monthly LLM inference spend. |
| Vendor Lock-in | Rewriting SDKs across 50+ microservices | Unified OpenAI-Compatible Endpoint : Switch providers in one config file. | 10x faster migration between AI vendors. |
Deploy the gateway in under 30 seconds:
git clone https://github.com/67spp/sentinel-ai-gateway.git
cd sentinel-ai-gateway
docker build -t sentinel-gateway .
docker run -p 8000:8000 sentinel-gateway
Once running, explore the interactive Swagger UI documentation at http://localhost:8000/docs.
1. Outbound Request (from your developer):
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"prompt": "Send invoice to ceo@enterprise.com using card 4532-1111-2222-3333",
"task_complexity": "auto"
}'
2. What the External Cloud LLM Actually Sees:
"Send invoice to [EMAIL_REDACTED_1] using card [CREDIT_CARD_REDACTED_2]"
3. What Sentinel Returns to Your Application:
{
"response": "Processed securely by local/llama-3-8b-instruct: Confirmed action for Send invoice to ceo@enterprise.com using card 4532-1111-2222-3333",
"sanitized_prompt": "Send invoice to [EMAIL_REDACTED_1] using card [CREDIT_CARD_REDACTED_2]",
"pii_detected_and_masked": 2,
"routed_model": "local/llama-3-8b-instruct",
"cache_hit": false,
"estimated_cost_saved_usd": 0.0045,
"latency_ms": 1.42
}
- Sub-2ms Regex & Pattern DLP Engine
- Reversible In-Memory Token Vault
- Complexity-based FinOps Routing
- Microsoft Presidio NLP integration for context-aware NER scrubbing
- Redis Vector DB integration for cosine-similarity semantic caching
- Role-Based Access Control (RBAC) & Prometheus/Grafana metrics export
Architected for high-compliance environments. For enterprise integration, custom deployment, or acquisition inquiries, reach out via LinkedIn.