Most teams start with a direct OpenAI (or Anthropic) SDK call. That works until you have three apps, two providers, and a security review asking who can call which model, at what rate, with what audit trail.
An AI / Agent Gateway sits between your apps (and agents) and every upstream model or tool. One place for routing, policy, and observability — without rewriting clients.
TrustGate is NeuralTrust’s open-source, security-first Agent Gateway (Go). It fronts LLM APIs and MCP tool servers behind Admin / Proxy / MCP planes, so you change a base URL and two headers instead of scattering keys and rate limits across services.
| Plane | Port | Job |
|---|---|---|
| Admin | ||
:8080 |
||
| Gateways, registries, consumers, auth, policies | ||
| Proxy | ||
:8081 |
||
| OpenAI-compatible LLM traffic + plugins | ||
| MCP | ||
:8082 |
||
| Aggregated Model Context Protocol endpoint for agents (e.g. Cursor) |
Policies (rate limit, token rate limit, request size, semantic cache, CORS) run in the data path before traffic hits providers.
Requires Docker (and git
). From a clean machine:
curl -fsSL https://raw.githubusercontent.com/NeuralTrust/TrustGate/main/scripts/install.sh | bash
Or, if you prefer the explicit path:
git clone https://github.com/NeuralTrust/TrustGate.git && cd TrustGate
cp .env.example .env
make up
Health checks:
curl localhost:8080/healthz # Admin
curl localhost:8081/healthz # Proxy
curl localhost:8082/healthz # MCP
With TrustGate up and OPENAI_API_KEY
set, the repo ships a script that creates a demo gateway, registers OpenAI, mints a consumer key, and sends a chat completion:
export OPENAI_API_KEY="sk-..."
./examples/curl-first-request/first-request.sh
From an app, keep the OpenAI SDK — point it at the proxy:
from openai import OpenAI
import os
client = OpenAI(
base_url="http://localhost:8081/my-app", # /{consumer_slug}
api_key="unused", # provider key lives in the gateway registry
default_headers={
"X-AG-Gateway-Slug": "demo",
"X-AG-API-Key": os.environ["CONSUMER_API_KEY"],
},
)
print(
client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Hello through TrustGate"}],
)
.choices[0]
.message.content
)
Full example: examples/openai-sdk/.
Agents shouldn’t each hold a private mesh of tool servers. TrustGate’s MCP plane (:8082
) aggregates registered MCP targets behind one endpoint with the same tenancy and policy model. Cursor setup notes live in examples/mcp-cursor/.
Choose a gateway when you need central policy (who / what / how fast), multi-provider routing with fallback, and increasingly MCP aggregation for agents — not only request logs.
If you mainly want a thin multi-provider proxy, LiteLLM is a strong developer default. If you mainly want analytics, Helicone leads with observability. TrustGate’s wedge is governance and security in the data path, including MCP.
Docs: docs.neuraltrust.ai · Repo: github.com/NeuralTrust/TrustGate
If this was useful, the repo is here — stars help others find it.
Disclosure: TrustGate / NeuralTrust DevRel.