MCP Gateway: What It Adds Over a Raw MCP Server

A developer detailed the design of an MCP gateway layer that sits between an agent host and a raw MCP server, adding per-key and per-team rate limits, budget caps, and one audit row per tool call. The writeup walks through middleware that carves out /api/mcp from the generic API rate limiter so tool traffic is metered by API key rather than IP, and an audit enrichment step that labels calls as "mcp" or "rest" at write time. The author argues a plain proxy cannot answer who called, how often, at what cost, or which log row proves it.

Short answer: An MCP gateway is the layer between an agent host and the MCP server. It adds what a raw server does not have: one upstream URL for every client, per-key and per-team request limits, budget caps clamped to the plan, and one audit row per tool call. A plain proxy forwards the same bytes and can describe none of it. Key takeaways tools/list and tools/call ; it has no opinion about who may call, how often, or at what cost. An MCP server answers questions about tools. A gateway answers questions about callers : which key is this, how many requests has it sent this minute, what has this team spent this month, and which row in the audit log proves the call happened. Those are four questions with four data sources, and none of them belong inside a tool implementation. The demand for that layer is measurable rather than assumed. The phrase carries roughly 2,400 US searches a month with a keyword difficulty of ai gateway at llm gateway at mcp proxy at mcp cli at SmartGate is one implementation of that layer — an MCP-native algorithm gateway for token control, traffic shaping, and agent audit. Hosts point at one Streamable HTTP endpoint and get seven tools: smart fetch, smart search, smart context gate, smart dedup, smart budget guard, smart memory and smart pipe. The twelve excerpts below are the parts of that path a raw MCP server simply does not have. The first decision any request meets is a path test at the edge, and the interesting part is what is excluded from it: middleware.ts — source lines 72–89 middleware async function middleware request: NextRequest { const { pathname } = request.nextUrl; // 1 API routes → rate-limit + feature gate always bypass intl/auth below if pathname.startsWith "/api/" { if pathname.startsWith "/api/webhooks/" && pathname.startsWith "/api/mcp" { const rateLimitResponse = await rateLimitApi pathname, request ; if rateLimitResponse return rateLimitResponse; const apiBlock = getDisabledApiResponse pathname ; if apiBlock return apiBlock; } return NextResponse.next ; } Three things are decided in eighteen lines. Traffic under /api/ is rate-limited and feature-gated before it reaches a handler — that is the generic API budget. Two prefixes are carved out of it: /api/webhooks/ for provider callbacks that must not be throttled, and /api/mcp , the tool-call path. The carve-out is the design statement of this whole page: MCP traffic is not charged against the API request budget, because it has its own limiter keyed by the API key rather than by IP. Anything outside /api/ falls through to the page logic below, so a tool call never touches locale or session middleware. A plain proxy has no equivalent branch — it applies one limit to every path, because it has no vocabulary for "this path is metered differently". Once a call is admitted it has to be attributable, and attribution happens at write time: backend/smartgate/core/audit enrichment.py — source lines 35–40 infer route def infer route path: str, route hdr: str - str: if route hdr: return route hdr if path.startswith "/mcp" : return "mcp" return "rest" Two lines of policy. A header wins if present, which lets an edge declare a route it knows better than the server can infer. With no header the path decides: anything under /mcp is labelled mcp , everything else rest . The fallback is a constant rather than an empty string, so old rows collect in one bucket instead of scattering across nulls. That matters more for an AI gateway than a classic one, because the audit row is part of what you are buying. A gateway that terminates both a REST surface and an MCP endpoint and labels neither produces logs that cannot answer "how much of this month's traffic was agent traffic". Deriving the label later from route tables is the version that breaks quietly, on the day a route is renamed. The limit that refuses a runaway agent is a pair of counters in a shared store — one per key, one per team: backend/smartgate/core/rate limiter.py — source lines 51–77 check rate limit mcp async def check rate limit mcp key id: str, team id: str, , per key limit: int, team ceiling: int, window seconds: int = 60, - dict: try: redis = await get redis except Exception as exc: logger.warning "mcp rate limit fail-open: %s", exc return {"allowed": True, "fail open": True} try: bucket = int time.time / window seconds key bucket = f"rate limit mcp:{key id}:{bucket}" team bucket = f"rate limit mcp team:{team id}:{bucket}" key count = await incr bucket redis, key bucket, window seconds if key count per key limit: ttl = await redis.ttl key bucket return { "allowed": False, "retry after": max 1, ttl , "limit scope": "mcp key", } The counting is deliberately boring: one integer per key per time bucket, incremented on entry and compared against the plan's number. What makes it diagnosable is the refusal payload — every rejection carries retry after from the bucket's own TTL and a limit scope naming which wall was hit, mcp key or mcp team . A client that gets a scoped 429 can back off correctly; one that gets a bare 429 retries in a loop and makes the incident worse. The second branch is the team ceiling, the one people forget to model: per-key limits alone mean a fleet of ten keys has ten times the allowance. The counters are incremented in sequence, key first, so a single key cannot consume more than its share of the team's. Sizing a bucket that several callers share at once is a concurrency decision rather than a rate decision, and Concurrency Control in AI Backend Systems https://smartgate.network/integration/concurrency-control-in-ai-backend-systems covers the queueing and pool side of it. Then the honest part. If the counter store cannot be reached, the function returns allowed: True with fail open: True rather than refusing every call in the fleet — an availability choice, and one that is visible in the return value instead of hidden in a log line. Rate limits and spend caps are different resources, and the second is computed from two objects rather than typed into a config file: lib/settings/clamp-gateway-policy.ts — source lines 4–35 clampGatewayPolicyToPlan function clampGatewayPolicyToPlan policy: GatewayPolicyV1, caps: PlanCapabilities, : GatewayPolicyV1 { const rpm = Math.min policy.playground.rpm, caps.maxPlaygroundRpm ; let dailyCap = policy.playground.daily request cap; if caps.minPlaygroundDailyCap = null { dailyCap = caps.minPlaygroundDailyCap; } else if dailyCap = null && caps.maxPlaygroundDailyCap = null { dailyCap = Math.min dailyCap, caps.maxPlaygroundDailyCap ; } return { ...policy, playground: { rpm, daily request cap: dailyCap, }, governance: { allow member tool overrides: caps.memberToolOverridesAllowed ? policy.governance.allow member tool overrides : false, allow integrator hmac budget: caps.l2HmacBudgetAllowed ? policy.governance.allow integrator hmac budget : false, }, }; } The shape is the useful part. Effective requests per minute is the smaller of what the team asked for and what the plan allows, so a policy written under a larger plan cannot survive a downgrade as a promised rate. The daily cap has three cases: a plan that sets a floor wins outright, otherwise the team's cap is clamped to the plan maximum, otherwise it passes through untouched. Two governance switches — member-level tool overrides and per-integrator HMAC budgets — are forced to false when the plan does not grant them. Everything here is pure: policy plus capabilities in, policy out, no store read. That makes it callable on the request path and in an admin screen with the same answer — and it is exactly the computation a traffic proxy has nowhere to put. The numbers the limiter enforces are read from one place, so a plan change does not become a code change: lib/plan-rate-limits.ts — source lines 5–12 getPlanRateLimits function getPlanRateLimits plan: Plan { const f = getPlanCatalogEntry plan .features; return { restWriteRpm: f.max team rpm, mcpRpmPerKey: f.mcp rpm per key, mcpRpmTeamCeiling: f.mcp rpm team ceiling, }; } Three values come out: the REST write ceiling for a team, the MCP rate per key and the MCP team ceiling. The asymmetry is deliberate — the protocol with autonomous callers gets two ceilings, because the caller that throttles itself politely and the caller that retries in a loop are the same caller at different hours. The REST side keeps one. Note where this lives: a catalog lookup, not a per-request computation. The plan catalog is the single source for what a plan may do, and the limiter only ever compares measured counts to these numbers. When the two disagree, the catalog wins, because it is the object a billing change edits. This is the shortest excerpt here and the one that separates a gateway from a proxy: lib/settings/load-gateway-policy.ts — source lines 10–12 loadGatewayPolicy function loadGatewayPolicy team: TeamPolicySource : GatewayPolicyV1 { return parseGatewayPolicy team.gatewayPolicy ; } Three lines: take the team record, parse its gateway policy, return it. The function is uninteresting; when it runs is the product. It runs on the request path, against the team the authenticated key resolved to, so the policy is per tenant and changes without a deploy or a restart. A proxy can do none of that. It sees bytes, a method and a destination, so the only limits it can enforce are the ones expressible in those terms — requests per second, bytes per second, IP allowlists. The moment the correct limit is "this key, this team, this month, in tokens", the proxy must be told by something that parsed the request. That something is the gateway, and this is where it reads its instructions. Policy objects arrive from a database column, which means they arrive in every shape a column can hold: lib/settings/plan-seeds.ts — source lines 15–21 isEmptyPolicy function isEmptyPolicy raw: unknown : boolean { if raw === null || raw === undefined return true; if typeof raw === "object" && Object.keys raw as object .length === 0 { return true; } return false; } Null, undefined and an empty object {} are one answer; anything else is another. The distinction the caller needs is not "was this falsy" but "has this team ever configured a policy" — because an empty policy is the normal state of a new team and a seed path can fill it, while a thrown error on the same input turns normal onboarding into a broken page. The three-line shape is also a small contract with the rest of the file: isEmptyPolicy is the only place that decides emptiness, so the seed logic and the admin form agree about when a team is unconfigured. Two different emptiness tests — one for the UI, one for the request path — is how a team ends up with a policy it can see but the gateway does not apply. Client configuration is where an MCP gateway earns the "one upstream URL" claim: lib/connect/mcp-config-templates.ts — source lines 249–264 mcpConfigFilename function mcpConfigFilename platform: McpPlatform : string { switch platform { case "Claude Desktop": return "claude desktop config.json"; case "Cursor": return "mcp.json"; case "Windsurf": return "mcp config.json"; case "Hermes": return "mcp.json"; case "OpenClaw": return "openclaw-smartgate-snippet.json"; default: return "smartgate-mcp.json"; } } A switch over the platform returning a filename per host — claude desktop config.json for Claude Desktop, mcp.json for Cursor, mcp config.json for Windsurf, a namespaced snippet for OpenClaw, and a default for anything else. The URL, the transport and the auth header are identical in every case; only the file to edit changes. That is the operational difference between a hosted gateway and a local stdio server. A stdio server is a process on the caller's machine: every host needs its own copy, its own runtime and its own upgrade path. A hosted endpoint is one URL plus one credential, and the host's config format is the only variable left. The function is small because the interesting engineering — transport, session handling, normalization — happens on the other side of that URL. That other side is worth reading once: actors, lifecycle and transport https://smartgate.network/industry/model-context-protocol-explained covers the handshake, the capability exchange, and why stdio and Streamable HTTP serve different deployments. The policy object is the security boundary, so writes to it are gated before they are validated: lib/settings/assert-plan-allows.ts — source lines 40–44 assertGatewayPolicyEditable function assertGatewayPolicyEditable caps: PlanCapabilities { if caps.teamGatewayPolicyEditable { throw new PlanFeatureDisabledError "PRO", "gateway policy" ; } } One guard, one error type. If the plan does not grant an editable team policy, the caller gets PlanFeatureDisabledError "PRO", "gateway policy" instead of a silent no-op — the error names both the missing capability and the protected object. The reasoning is worth stating plainly, because "who can edit the limits" decides whether the limits mean anything. Per-key limits, budget caps and governance switches all live in one object the request path reads, so any member who can write that object can raise their own ceiling and turn enforcement into a suggestion. The write path is therefore plan-gated, and the check runs before the mutation rather than after: a rejected edit leaves the previous policy in force, which is the fail-closed direction a security control should fail in. Tool registration is declarative, and the interesting claim is about the function body: backend/smartgate/api/mcp.py — source lines 128–148 smart search @server.tool name="smart search", description=TOOL DESCRIPTIONS "smart search" , annotations=tool annotations "smart search" , async def smart search query: str = Field description="Search query string." , max results: int = Field default=10, description="Maximum number of results to return 1–50 .", , - str: , registry = app state module = registry.get "search" ctx = tool ctx return await run with audit "search", ctx, module.process ctx, query=query, max results=max results , {"query": query}, Each tool is declared once, with its description read from a shared table and its annotations derived from the tool name, so the model-facing metadata and the documentation cannot drift apart. The body then does what every other tool does: resolve the module from the registry, build a call context, run the coroutine, and return through run with audit with the arguments that should be recorded. That shared ending is the gateway's actual product. Because all seven tools leave through one function, a missing audit row means the call never reached the gateway rather than that one tool forgot to log — and there is one place to hold the budget check, the error translation and the trace id. A raw MCP server hands you the tool surface and leaves that endpoint design to you. Every limit on this page is a counter in a store, and the store is resolved rather than assumed: lib/redis/config.ts — source lines 26–29 isTcpRedisConfigured function isTcpRedisConfigured : boolean { const url = redisUrl ; return Boolean url && url.startsWith "redis://" || url.startsWith "rediss://" ; } A URL test, two schemes, one boolean. redis:// and rediss:// are the TCP forms; anything else — including the HTTP-based serverless Redis some deployments use instead — is not what this check asks about. Keeping the test narrow lets the caller tell "no counter store is configured" apart from "a counter store is configured in a shape this path does not speak". The client-side consequence is why this matters on a page about gateways. Whatever the host is — a VS Code extension, a desktop app, a CLI agent — the counter it is measured against lives on the server. Two hosts calling the same key share one budget, limits survive a host restart, and reinstalling the client does not grant a fresh allowance. Local enforcement cannot promise any of that. Resolution order matters when the store is missing, because it decides whether limits hold: lib/redis/config.ts — source lines 38–42 resolveRedisMode function resolveRedisMode : RedisMode { if isUpstashRestConfigured return "upstash"; if isTcpRedisConfigured return "tcp"; return "none"; } Two checks in priority order and a constant: an Upstash-style REST configuration wins, then a TCP URL, then none . The first two are live limiters; the third is the mode in which the limiter's fail-open branch is the only branch that runs, and every call is admitted. State that as a limitation rather than a feature. A remote MCP server with no counter store still answers — availability first — but its per-key limits become advisory and the budget guard has nothing to count against. The audit rows still appear, which is the useful part: a deployment that degraded to none can be found afterwards from what the rows do not contain, instead of from a spend surprise. Each alternative is described the way its own documentation describes it; the links are sources, not claims from this page. Vendors describe this layer with different scopes, and the difference is instructive. Microsoft's MCP Gateway project calls itself "a reverse proxy and management layer for MCP servers", aimed at session-aware routing and lifecycle management Microsoft https://microsoft.github.io/mcp-gateway/ . Kong describes an MCP gateway as centralising "access, security, and management for multiple Model Context Protocol servers" Kong https://konghq.com/blog/learning-center/what-is-a-mcp-gateway . Both descriptions are consistent with this page, and both emphasise the routing and lifecycle half — the half a well-built proxy can reach. The half these excerpts show is per-caller accounting: which key called, how often, at what spend, and the row that records it. | | What it governs | Where the limits live | What you pay | |---|---|---|---| | A raw MCP server stdio | The tools it implements; nothing about the caller | Nowhere — the process trusts the host that launched it | Host machine time and your own engineering | | An MCP server behind a plain reverse proxy | Forwarding, TLS, maybe an IP-level request cap | In the proxy configuration, blind to tool calls | The proxy tier, plus the metering you still build | | A model-proxy AI gateway Cloudflare https://developers.cloudflare.com/ai-gateway/ , Kong https://developer.konghq.com/ai-gateway/ | Model traffic, per each vendor's own documentation | In the vendor's console or the proxy's config file | A subscription or a per-token fee; agent tool calls often stay outside its view | | SmartGate — an MCP-native algorithm gateway | Which tools a key may call, how fast, how much it may spend, and the row that proves each call | In one policy object, read per request and clamped to the plan | Platform fee, plus a share only after measured savings pass the threshold | The billing model is the part worth testing against your own invoices: pay for the platform, share only when you save . Free is $0 with 2M tokens a month, all seven tools, 120 MCP requests a Does an MCP gateway replace the MCP server? No. The server owns the tools — what they do, what arguments they take, what they return. The gateway owns the caller: identity, rate, budget and the audit record. Removing the gateway leaves a working server with no enforcement; removing the server leaves a gateway with nothing to route. Is an MCP gateway the same thing as an MCP proxy? A proxy moves bytes between a client and a server. A gateway parses enough of the request to price it, attribute it and refuse it. Every gateway can proxy; the reverse is not true, because per-key and per-month limits need fields a byte pipe never reads. Where do the per-key limits actually apply? In the gateway, against the key that authenticated the request, before the tool runs. Two hosts sharing one key share one allowance, and the team ceiling is checked after the per-key window, so one key cannot consume the whole team's budget in a burst. What happens to limits if the counter store goes down? The limiter fails open: calls are admitted with a flag naming the degraded path. That is a choice for availability over enforcement, and it is visible in the response rather than buried in logs — so a degraded window is something you can find afterwards. Do budget caps need a code change when a plan changes? No. The effective rate and daily cap are computed as the minimum of the team policy and the plan capability, and the governance switches are forced off when the plan does not grant them. Editing the plan catalog moves every team bound by it. Which clients work without a local server process? Any host that speaks Streamable HTTP over a URL. The gateway ships per-host configuration blocks for the common ones, but the transport is the standard one — nothing here requires a process on the caller's machine. rest label. The code in this article is not transcribed. Each block was cut directly out of the slice body returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body before publication; the first line inside every fence records the file and the exact source lines. Symbols were pinned by whole-name containment rule A level 2 and confirmed by the service's slot-proof endpoint before any prose was written. Ten of the twelve excerpts are whole slice bodies; the middleware and check rate limit mcp blocks are windows into longer files, and the lines around them are described rather than quoted. Demand figures come from this project's own keyword run, recorded in research brief.md and search volume.json : the section phrases ai gateway 2,400 , llm gateway 1,600 , mcp proxy 720 , mcp cli 480 , ai agent security 480 , openclaw mcp 480 , vscode mcp 390 , remote mcp server 390 , mcp router 260 , model gateway 70 , api gateway for ai 20 and mcp firewall 20 , each with its competition band; the head term and its difficulty were measured in the same research pass. | | SERP keyword | Symbol | File | Source lines | How it was pinned | sha256 12 | |---|---|---|---|---|---|---| | 1 | mcp router | middleware | middleware.ts | 72–89 | rule A L2 → slot-proof | 9ab198bee98b | | 2 | api gateway for ai | infer route | backend/smartgate/core/audit enrichment.py | 35–40 | rule A L2 → slot-proof | 8cb200979cc6 | | 3 | mcp firewall | check rate limit mcp | backend/smartgate/core/rate limiter.py | 51–77 | rule A L2 → slot-proof | b540f8154beb | | 4 | ai gateway | clampGatewayPolicyToPlan | lib/settings/clamp-gateway-policy.ts | 4–35 | rule A L2 → slot-proof | 613facc8545b | | 5 | llm gateway | getPlanRateLimits | lib/plan-rate-limits.ts | 5–12 | rule A L2 → slot-proof | 6d29a819fd91 | | 6 | mcp proxy | loadGatewayPolicy | lib/settings/load-gateway-policy.ts | 10–12 | rule A L2 → slot-proof | 82918a7047cc | | 7 | model gateway | isEmptyPolicy | lib/settings/plan-seeds.ts | 15–21 | rule A L2 → slot-proof | 142457b5325c | | 8 | mcp cli | mcpConfigFilename | lib/connect/mcp-config-templates.ts | 249–264 | rule A L2 → slot-proof | d60503ebfb68 | | 9 | ai agent security | assertGatewayPolicyEditable | lib/settings/assert-plan-allows.ts | 40–44 | rule A L2 → slot-proof | 5bf979730866 | | 10 | openclaw mcp | smart search | backend/smartgate/api/mcp.py | 128–148 | rule A L2 → slot-proof | 12366ba0d241 | | 11 | vscode mcp | isTcpRedisConfigured | lib/redis/config.ts | 26–29 | rule A L2 → slot-proof | 7ca086a475d5 | | 12 | remote mcp server | resolveRedisMode | lib/redis/config.ts | 38–42 | rule A L2 → slot-proof | 3a9dd8e9daf4 | Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before publication. 12 of 12 sections pinned, 0 abstentions, 0 misses.