{"slug": "agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic", "title": "Agentgateway – One high-performance gateway for service, LLM, and MCP traffic", "summary": "Agentgateway launched as an open source HTTP and gRPC gateway that routes service, LLM provider, and Model Context Protocol (MCP) and agent-to-agent (A2A) traffic through a single data plane and binary. The gateway fronts OpenAI, Anthropic, Bedrock, Gemini, Vertex, and self-hosted Llama inference with a drop-in OpenAI-compatible API, adding token budgets, semantic caching, prompt redaction, and latency- and cost-aware routing across vLLM, TGI, and Triton backends. It ships with mTLS, OIDC, JWT validation, per-consumer API keys, OpenTelemetry tracing, and tool-call inspection intended to block prompt injection before it reaches an agent or MCP server.", "body_md": "One high-performance gateway for service, LLM, and MCP traffic.\n\nAn open source HTTP and gRPC gateway that handles traditional application traffic and AI-native protocols in one data plane. Route, secure, observe, and govern services, LLM provider traffic, MCP tools, and agent-to-agent communication without stitching together separate gateways.\n\n`curl -sL https://agentgateway.dev/install | bash`\n### LLM Gateway\n\nRoute to OpenAI, Claude, Gemini, or any self-hosted model with credentialing and failover.\n\n### MCP / A2A\n\nNative protocol support for Model Context Protocol and agent-to-agent traffic.\n\n### APIs & Services\n\nHTTP, gRPC, and TCP traffic — observable by default with mTLS and OIDC built in.\n\nSwipe to explore →\n\n## Everything you need for agent traffic — in one binary.\n\nOne control plane for service traffic, LLM routing, MCP, A2A, and inference — with the policy, observability, and cost controls that a platform team actually trusts.\n\n### ROUTE\n\nOne data plane for HTTP, gRPC, and TCP services, every major LLM provider, self-hosted inference, and the MCP and A2A protocols your agents speak.\n\nRun any north–south or east–west traffic through a single binary. Zero-config TLS, OIDC, mTLS-rotation, native Envoy compatibility for those who want it.\n\nDrop-in OpenAI-compatible API in front of OpenAI, Anthropic, Bedrock, Gemini, Vertex, and OSS Llama runs — with token budgets, semantic caching, and prompt redaction.\n\nLatency-aware, cost-aware, model-aware routing for self-hosted inference. Plug-in vLLM, TGI, Triton; agentgateway picks the warmest replica.\n\nTreat MCP servers like microservices: discover them, sign and scope every tool call, and keep an audit trail of what agents invoked.\n\nRoute agent-to-agent invocations between LangChain, CrewAI, ADK, and your own runtime, with identity and tracing on every hop.\n\n### SECURE\n\nAuthenticate, authorize, and redact on every hop — not just at the edge.\n\nThe policy layer itself: what attaches where, how rules merge, and how a decision is reached on every hop — with tamper-evident audit logs.\n\nValidate JWTs at the edge and pull claims into routing and authorization decisions, or hand off a full OIDC login flow for browser traffic.\n\nIssue and rotate API keys per consumer, or fall back to basic auth for internal tooling that cannot speak OIDC.\n\nDelegate the allow/deny call to your own service over gRPC or HTTP, so existing entitlement logic stays where it already lives.\n\nExpress allow/deny rules over method, path, headers, and JWT claims — evaluated per hop, on both HTTP and network listeners.\n\nAuthenticate callers into MCP servers and scope which tools each identity may invoke, so an agent cannot reach past its mandate.\n\nTerminate TLS at the listener and originate verified TLS or mTLS to every backend, with certificate rotation handled for you.\n\nBrowser-facing protections for gateways that serve web apps directly: origin allow-lists and CSRF token enforcement.\n\n### OBSERVE\n\nKnow exactly what happened on every request, and debug it without a restart.\n\nOpenTelemetry by default. Token-usage histograms on Prometheus, traces in Jaeger, and request logs with status, latency, and token counts — each carrying the realized USD cost.\n\nSee what a proxy actually loaded, without a restart. `proxy trace` taps live requests into an interactive terminal view, `proxy config` dumps runtime state, and log levels change on the fly.\n\n### GOVERN\n\nDecide what each caller may invoke, inspect what flows through, and know how overlapping rules resolve.\n\nDecide which tools on which MCP targets a caller may reach, and rewrite or restrict what the server exposes downstream.\n\nInspect tool calls and responses in flight, blocking prompt injection and unsafe payloads before they reach an agent or a server.\n\nPattern and model-based checks on LLM prompts and completions, with redaction for anything that should never leave your estate.\n\n### COST\n\nAttribute, price, and cap what your agents spend.\n\nSpend control in three stages: attribute it, price it, then enforce it — before the invoice is the first place you find out.\n\nHard caps per key or team: cut off token consumption and dollar spend once a limit is hit, rather than alerting after the fact.\n\nIssue scoped keys per team, app, or user so every request carries an owner — the unit that budgets and reports attach to.\n\nSpend, token usage, and traffic in the admin UI, grouped by model, provider, and user, so you can see where the money went.\n\nPer-model input and output pricing, so each request carries a realized USD cost that logs, metrics, traces, and policies can all read.\n\n## Getting started\n\n[View all docs →](https://agentgateway.dev/docs/)\n\n#### Quick start\n\nGet started with agentgateway in less than 5 minutes — local-first.\n\n#### MCP connectivity guide\n\nConnect agentgateway to MCP servers with built-in discovery and auth.\n\n#### Deployment options\n\nBinary, Docker, or Kubernetes — pick what fits your stack.\n\n#### Hands-on lab: LLM cost management\n\nFree, on-demand lab — monitor LLM usage, apply consumption controls, and optimize costs with agentgateway.\n\n## Popular integrations\n\n### What could be improved?\n\nYour feedback helps us improve assistant answers and identify docs gaps we should fix.\n\nNeed more help? Join us on Discord:\n[https://discord.gg/y9efgEmppm](https://discord.gg/y9efgEmppm)\n\nWant to use your own agent? Add the Solo MCP server to query our docs directly. Get started here:\n[https://search.solo.io/](https://search.solo.io/).", "url": "https://wpnews.pro/news/agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic", "canonical_source": "https://agentgateway.dev/", "published_at": "2026-10-18 11:20:55+00:00", "updated_at": "2026-10-18 11:48:22.022711+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-infrastructure", "ai-tools", "large-language-models"], "entities": ["agentgateway", "OpenAI", "Anthropic", "Google Gemini", "Amazon Bedrock", "Vertex", "LangChain", "CrewAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic", "markdown": "https://wpnews.pro/news/agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic.md", "text": "https://wpnews.pro/news/agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic.txt", "jsonld": "https://wpnews.pro/news/agentgateway-one-high-performance-gateway-for-service-llm-and-mcp-traffic.jsonld"}}