Show HN: Self-hosted AI gateway – MCP, budget, PII, smart router, fallback ILTER, a self-hosted AI gateway, launches as a single static executable that sits between applications and AI providers to address cost control, data privacy, and API reliability. The gateway features an MCP marketplace, smart routing with real-time complexity scoring, budget limits, PII masking, semantic caching, prompt guardrails, and agent loop detection, requiring no code changes or external dependencies. Deploying AI models to production brings critical operational challenges such as unpredictable costs, data privacy risks, and API outages. ILTER is an independent gateway that sits between your application and AI providers; transforming your entire AI traffic into a secure, optimized, and fully controlled infrastructure without requiring any architectural changes in your code. - 🧩 MCP Gateway & Marketplace: Connect AI to your APIs, CRM, or MCP servers — zero changes required. - ⚡ Smart Router: Every request is scored for complexity in real time and routed to the right model tier. - 🌐 Smart Fallback: Provider down or rate-limited? ILTER fails over to another key or provider, no changes. - 💰 Budget Control: Hard daily/monthly spending limits per key — traffic cuts off the instant a limit is hit. - 🪪 PII Guard: Emails, SSNs, and other sensitive data are masked before they ever leave your network. - 🧠 Semantic Cache: Queries are served from cache — vector search with SHA-256 exact-match fallback. - 🛡️ Prompt Guardrails: Blocks injection, toxic content, and off-topic requests before they reach the model. - 🌀 Agent Loop Detector: Catches loops by rate, fingerprint, cost, or depth before burning budget. - ⏱️ Cron Engine: Schedule recurring AI workflows with standard cron expressions — no external queue needed. - 📡 Observability: Prometheus metrics and OpenTelemetry tracing, ready for Grafana, Datadog, or Honeycomb. - 📊 Dashboard: Embedded web UI for KPIs, audit logs, API keys, and built-in chat playground — no server. All this infrastructure requires no external server, Node.js/Python environment, or complex dependencies. It comes as a zero-setup, single static executable binary ready to run in seconds. ./ilter serve Proxy: http://localhost:8181/v1/chat/completions Dashboard: http://localhost:9191 Metrics: http://localhost:9192/metrics Every business integrating AI faces uncontrolled costs, privacy concerns, and unpredictable agent behaviors. ILTER solves these directly at the gateway level. Connecting AI to your internal APIs or CRM usually requires heavy client-side modifications. Zero Client Changes: Automatically injects tools from registered MCP servers into every chat completion request. Interception: ILTER catches the tool call , executes the MCP server tool locally stdio /SSE , and returns the result to the model. OpenAPI Bridge: Convert any REST API to an MCP tool list instantly. OAuth PKCE Support: Standard OAuth PKCE authorization endpoints /.well-known/ , /authorize , /token , /register on port 8181 for remote MCP clients. Using expensive models for every simple question wastes engineering time and money. ILTER routes requests dynamically based on context. Real-Time Scoring: <0.03ms heuristic complexity score 0–100 based on word count, reasoning-required phrases, code blocks, and tool calls. Automatic Tier Selection: Routes dynamically between economy, standard, and premium models. Custom Overrides: E.g., complexity 50 → gpt-4o or prompt contains "analyze" → claude-sonnet . Load Balancing: Weighted round-robin, cost-optimized, latency-optimized, and priority-based strategies across healthy provider circuits. Don't get locked into a single vendor or disrupted by provider outages. Single Endpoint: Hides 9 providers behind one uniform OpenAI-compatible API. Circuit Breaker: RetryTransport → CircuitBreakerTransport . 5 consecutive failures open the circuit and trigger automatic fallback. Seamless Failover: Switching from OpenAI to Anthropic during an outage requires zero code changes. Vector Search: Redis Stack-backed similarity search to catch and serve repetitive queries. Exact-Match Fallback: SHA-256 caching works even without an embedding model. Async Operations: Cache writing does not block the primary request path. Security: Prompt injection detection and toxicity filtering. Topic Control: Keyword-based and custom regex rules to block specific topics. Actions: Set per-rule severity to block , warn , or mask . Per-token pricing creates end-of-month surprises. ILTER tracks costs in real-time for each API key and enforces hard limits. Hard Kill Switch: Configurable daily and monthly spending limits USD . Returns 429 budget exceeded the millisecond the limit is breached. Warning Thresholds: Default 80% usage alerts. Real-time Tracking: Cost breakdowns by model, provider, and individual keys via the dashboard. Sending customer data to cloud providers violates GDPR and HIPAA. ILTER masks this data before it ever leaves your network. Triple-Layer Engine: Bloom Filter + Aho-Corasick Trie + Regex working together. Detected Types: Names EN/TR , Emails, Phones, SSN/National IDs TCKN , Credit Cards Luhn validated , IPs. Three Modes: Mask PII EMAIL , Reversible PII:EMAIL:a8b9f1 — restored in response/SSE stream , or Block 422 pii blocked . Ultra-Fast: <0.04ms latency overhead with <5MB memory footprint. Agentic workflows can enter infinite loops, generating massive bills overnight. ILTER catches them while you sleep. Rate Limit: Blocks if 30 requests/sec. Fingerprint Match: Blocks if the exact same prompt is sent 5 times in a 20-request window. Cost Accumulation: Blocks if $5 is spent within 5 minutes on a single session. Session Depth: Throttles if a single session exceeds 100 requests. Running periodic AI tasks summaries, reports usually requires external queues and workers. ILTER embeds this directly in the binary. No External Queue: Uses standard cron expressions e.g., 0 9 1 . Workflow State: Pass data between steps using {{.Input}} and {{.prev}} . Robust Execution: Webhook triggers, exponential backoff, and dead-letter queues included. Metrics: Native Prometheus metrics on /metrics Port 9192 . Tracing: OpenTelemetry OTLP push ready for Grafana Cloud, Datadog, or Honeycomb. Audit Logs: Full request logs, cost trends, and PII event indicators. No npm, no build step, no separate server. A modern web UI embedded directly into the Go binary via embed . Overview: KPIs, monthly spend, routing decisions, and feature toggle hub. Chat Playground: Use any registered model directly from the dashboard. Management: Create API keys, configure thresholds, register MCP servers, and manage Cron jobs. Accessible at Port 9191 . Requires nothing — it just works ./ilter serve For the interactive setup wizard: ./ilter init To see the dashboard filled with mock costs, requests, and PII events: ./ilter init --demo && ./ilter serve Your first real request Point your existing OpenAI SDK to ILTER : curl -X POST http://localhost:8181/v1/chat/completions \ -H "Authorization: Bearer