If your problem is "one OpenAI-compatible endpoint for 50 providers," a proxy like **LiteLLM** is usually the right default. Huge catalog, huge community, good enough ops for most teams.
If your problem is "agents that call tools (MCP) with the same identity, audit, budgets, and policy as model calls," a thinner **Agent Gateway** starts to matter — one control plane so auth doesn't drift between chat completions and tool traffic.
| Need | Usually start with | Why |
|---|---|---|
| Broadest provider catalog, ecosystem glue | LiteLLM | Depth beats novelty |
| Hosted control plane, zero ops | Portkey / Helicone-class | Ops off your plate |
| Self-host, low overhead Go proxy | Bifrost / similar | Latency + governance focus |
| Self-host LLM + MCP same policy/identity | ||
| Agent Gateway (e.g. TrustGate) | One plane for models + tools | |
| K8s/service-mesh native | Envoy AI Gateway / Kong | Fits existing mesh |
In practice the useful model is:
tools/list
is filtered per consumertools/call
, so spend and audit line up.That is the wedge: not "another 100 providers," but one aggregation plane for models + tools.
Repo: https://github.com/NeuralTrust/TrustGate Built in Go by NeuralTrust. Quickstart and "Why TrustGate" live in the README — including where it is not the right pick vs LiteLLM/Helicone/Portkey.
Disclosure: TrustGate / NeuralTrust DevRel.
source & further reading
dev.to — original article
483 tests passed, but Vestibule RAG framework wasn't installable — lessons from building with AI agents
Why AI Virtual Staging Needs Constraints More Than It Needs More Creativity
skillcheck Update: Scorer Fixes, Cleaner Failures, Honest Token Numbers