Short answer: An MCP gateway is the layer between an agent host and the MCP server. It adds
what a raw server does not have: one upstream URL for every client, per-key and per-team
request limits, budget caps clamped to the plan, and one audit row per tool call. A plain proxy
forwards the same bytes and can describe none of it.
Key takeaways
tools/list and
tools/call; it has no opinion about who may call, how often, or at what cost.
An MCP server answers questions about tools. A gateway answers questions about callers: which key
is this, how many requests has it sent this minute, what has this team spent this month, and which
row in the audit log proves the call happened. Those are four questions with four data sources, and
none of them belong inside a tool implementation.
The demand for that layer is measurable rather than assumed. The phrase carries roughly 2,400 US searches a month with a keyword difficulty of
ai gateway atllm gateway at mcp proxy at mcp cli at SmartGate is one implementation of that layer — an MCP-native algorithm gateway for token control,
traffic shaping, and agent audit. Hosts point at one Streamable HTTP endpoint and get seven tools:
smart_fetch, smart_search, smart_context_gate, smart_dedup, smart_budget_guard, smart_memory and
smart_pipe. The twelve excerpts below are the parts of that path a raw MCP server simply does not
have.
The first decision any request meets is a path test at the edge, and the interesting part is what
is excluded from it:
async function middleware(request: NextRequest) {
const { pathname } = request.nextUrl;
// 1) API routes → rate-limit + feature gate (always bypass intl/auth below)
if (pathname.startsWith("/api/")) {
if (
!pathname.startsWith("/api/webhooks/") &&
!pathname.startsWith("/api/mcp")
) {
const rateLimitResponse = await rateLimitApi(pathname, request);
if (rateLimitResponse) return rateLimitResponse;
const apiBlock = getDisabledApiResponse(pathname);
if (apiBlock) return apiBlock;
}
return NextResponse.next();
}
Three things are decided in eighteen lines. Traffic under /api/ is rate-limited and
feature-gated before it reaches a handler — that is the generic API budget. Two prefixes are
carved out of it: /api/webhooks/ for provider callbacks that must not be throttled, and
/api/mcp, the tool-call path. The carve-out is the design statement of this whole page: MCP
traffic is not charged against the API request budget, because it has its own limiter keyed by the
API key rather than by IP. Anything outside /api/ falls through to the page logic below, so a
tool call never touches locale or session middleware. A plain proxy has no equivalent branch — it
applies one limit to every path, because it has no vocabulary for "this path is metered
differently".
Once a call is admitted it has to be attributable, and attribution happens at write time:
def infer_route(path: str, route_hdr: str) -> str:
if route_hdr:
return route_hdr
if path.startswith("/mcp"):
return "mcp"
return "rest"
Two lines of policy. A header wins if present, which lets an edge declare a route it knows better
than the server can infer. With no header the path decides: anything under /mcp is labelled
mcp, everything else rest. The fallback is a constant rather than an empty string, so old rows
collect in one bucket instead of scattering across nulls.
That matters more for an AI gateway than a classic one, because the audit row is part of what you
are buying. A gateway that terminates both a REST surface and an MCP endpoint and labels neither
produces logs that cannot answer "how much of this month's traffic was agent traffic". Deriving
the label later from route tables is the version that breaks quietly, on the day a route is
renamed.
The limit that refuses a runaway agent is a pair of counters in a shared store — one per key, one
per team:
async def check_rate_limit_mcp(
key_id: str,
team_id: str,
*,
per_key_limit: int,
team_ceiling: int,
window_seconds: int = 60,
) -> dict:
try:
redis = await get_redis()
except Exception as exc:
logger.warning("mcp rate_limit fail-open: %s", exc)
return {"allowed": True, "fail_open": True}
try:
bucket = int(time.time() / window_seconds)
key_bucket = f"rate_limit_mcp:{key_id}:{bucket}"
team_bucket = f"rate_limit_mcp_team:{team_id}:{bucket}"
key_count = await _incr_bucket(redis, key_bucket, window_seconds)
if key_count > per_key_limit:
ttl = await redis.ttl(key_bucket)
return {
"allowed": False,
"retry_after": max(1, ttl),
"limit_scope": "mcp_key",
}
The counting is deliberately boring: one integer per key per time bucket, incremented on entry and
compared against the plan's number. What makes it diagnosable is the refusal payload — every
rejection carries retry_after from the bucket's own TTL and a limit_scope naming which wall
was hit, mcp_key or mcp_team. A client that gets a scoped 429 can back off correctly; one that
gets a bare 429 retries in a loop and makes the incident worse.
The second branch is the team ceiling, the one people forget to model: per-key limits alone mean a
fleet of ten keys has ten times the allowance. The counters are incremented in sequence, key
first, so a single key cannot consume more than its share of the team's.
Sizing a bucket that several callers share at once is a concurrency decision rather than a rate
decision, and Concurrency Control in AI Backend Systems
covers the queueing and pool side of it.
Then the honest part. If the counter store cannot be reached, the function returns allowed: True
with fail_open: True rather than refusing every call in the fleet — an availability choice, and
one that is visible in the return value instead of hidden in a log line.
Rate limits and spend caps are different resources, and the second is computed from two objects
rather than typed into a config file:
function clampGatewayPolicyToPlan(
policy: GatewayPolicyV1,
caps: PlanCapabilities,
): GatewayPolicyV1 {
const rpm = Math.min(policy.playground.rpm, caps.maxPlaygroundRpm);
let dailyCap = policy.playground.daily_request_cap;
if (caps.minPlaygroundDailyCap != null) {
dailyCap = caps.minPlaygroundDailyCap;
} else if (
dailyCap != null &&
caps.maxPlaygroundDailyCap != null
) {
dailyCap = Math.min(dailyCap, caps.maxPlaygroundDailyCap);
}
return {
...policy,
playground: {
rpm,
daily_request_cap: dailyCap,
},
governance: {
allow_member_tool_overrides: caps.memberToolOverridesAllowed
? policy.governance.allow_member_tool_overrides
: false,
allow_integrator_hmac_budget: caps.l2HmacBudgetAllowed
? policy.governance.allow_integrator_hmac_budget
: false,
},
};
}
The shape is the useful part. Effective requests per minute is the smaller of what the team asked
for and what the plan allows, so a policy written under a larger plan cannot survive a downgrade
as a promised rate. The daily cap has three cases: a plan that sets a floor wins outright,
otherwise the team's cap is clamped to the plan maximum, otherwise it passes through untouched.
Two governance switches — member-level tool overrides and per-integrator HMAC budgets — are forced
to false when the plan does not grant them.
Everything here is pure: policy plus capabilities in, policy out, no store read. That makes it
callable on the request path and in an admin screen with the same answer — and it is exactly the
computation a traffic proxy has nowhere to put.
The numbers the limiter enforces are read from one place, so a plan change does not become a code
change:
function getPlanRateLimits(plan: Plan) {
const f = getPlanCatalogEntry(plan).features;
return {
restWriteRpm: f.max_team_rpm,
mcpRpmPerKey: f.mcp_rpm_per_key,
mcpRpmTeamCeiling: f.mcp_rpm_team_ceiling,
};
}
Three values come out: the REST write ceiling for a team, the MCP rate per key and the MCP team
ceiling. The asymmetry is deliberate — the protocol with autonomous callers gets two ceilings,
because the caller that throttles itself politely and the caller that retries in a loop are the
same caller at different hours. The REST side keeps one.
Note where this lives: a catalog lookup, not a per-request computation. The plan catalog is the
single source for what a plan may do, and the limiter only ever compares measured counts to these
numbers. When the two disagree, the catalog wins, because it is the object a billing change edits.
This is the shortest excerpt here and the one that separates a gateway from a proxy:
function loadGatewayPolicy(team: TeamPolicySource): GatewayPolicyV1 {
return parseGatewayPolicy(team.gatewayPolicy);
}
Three lines: take the team record, parse its gateway policy, return it. The function is
uninteresting; when it runs is the product. It runs on the request path, against the team the
authenticated key resolved to, so the policy is per tenant and changes without a deploy or a
restart.
A proxy can do none of that. It sees bytes, a method and a destination, so the only limits it can
enforce are the ones expressible in those terms — requests per second, bytes per second, IP
allowlists. The moment the correct limit is "this key, this team, this month, in tokens", the
proxy must be told by something that parsed the request. That something is the gateway, and this
is where it reads its instructions.
Policy objects arrive from a database column, which means they arrive in every shape a column can
hold:
function isEmptyPolicy(raw: unknown): boolean {
if (raw === null || raw === undefined) return true;
if (typeof raw === "object" && Object.keys(raw as object).length === 0) {
return true;
}
return false;
}
Null, undefined and an empty object {} are one answer; anything else is another. The
distinction the caller needs is not "was this falsy" but "has this team ever configured a policy"
— because an empty policy is the normal state of a new team and a seed path can fill it, while a
thrown error on the same input turns normal onboarding into a broken page.
The three-line shape is also a small contract with the rest of the file: isEmptyPolicy is the
only place that decides emptiness, so the seed logic and the admin form agree about when a team is
unconfigured. Two different emptiness tests — one for the UI, one for the request path — is how a
team ends up with a policy it can see but the gateway does not apply.
Client configuration is where an MCP gateway earns the "one upstream URL" claim:
function mcpConfigFilename(platform: McpPlatform): string {
switch (platform) {
case "Claude Desktop":
return "claude_desktop_config.json";
case "Cursor":
return "mcp.json";
case "Windsurf":
return "mcp_config.json";
case "Hermes":
return "mcp.json";
case "OpenClaw":
return "openclaw-smartgate-snippet.json";
default:
return "smartgate-mcp.json";
}
}
A switch over the platform returning a filename per host — claude_desktop_config.json for Claude
Desktop, mcp.json for Cursor, mcp_config.json for Windsurf, a namespaced snippet for OpenClaw,
and a default for anything else. The URL, the transport and the auth header are identical in every
case; only the file to edit changes.
That is the operational difference between a hosted gateway and a local stdio server. A stdio
server is a process on the caller's machine: every host needs its own copy, its own runtime and
its own upgrade path. A hosted endpoint is one URL plus one credential, and the host's config
format is the only variable left. The function is small because the interesting engineering —
transport, session handling, normalization — happens on the other side of that URL. That other side
is worth reading once: actors, lifecycle and transport
covers the handshake, the capability exchange, and why stdio and Streamable HTTP serve different
deployments.
The policy object is the security boundary, so writes to it are gated before they are validated:
function assertGatewayPolicyEditable(caps: PlanCapabilities) {
if (!caps.teamGatewayPolicyEditable) {
throw new PlanFeatureDisabledError("PRO", "gateway_policy");
}
}
One guard, one error type. If the plan does not grant an editable team policy, the caller gets
PlanFeatureDisabledError("PRO", "gateway_policy") instead of a silent no-op — the error names
both the missing capability and the protected object.
The reasoning is worth stating plainly, because "who can edit the limits" decides whether the
limits mean anything. Per-key limits, budget caps and governance switches all live in one object
the request path reads, so any member who can write that object can raise their own ceiling and
turn enforcement into a suggestion. The write path is therefore plan-gated, and the check runs
before the mutation rather than after: a rejected edit leaves the previous policy in force, which
is the fail-closed direction a security control should fail in.
Tool registration is declarative, and the interesting claim is about the function body:
@server.tool(
name="smart_search",
description=TOOL_DESCRIPTIONS["smart_search"],
annotations=tool_annotations("smart_search"),
)
async def smart_search(
query: str = Field(description="Search query string."),
max_results: int = Field(
default=10,
description="Maximum number of results to return (1–50).",
),
) -> str:
_, registry = _app_state()
module = registry.get("search")
ctx = _tool_ctx()
return await _run_with_audit(
"search",
ctx,
module.process(ctx, query=query, max_results=max_results),
{"query": query},
)
Each tool is declared once, with its description read from a shared table and its annotations
derived from the tool name, so the model-facing metadata and the documentation cannot drift apart.
The body then does what every other tool does: resolve the module from the registry, build a call
context, run the coroutine, and return through _run_with_audit with the arguments that should be
recorded.
That shared ending is the gateway's actual product. Because all seven tools leave through one
function, a missing audit row means the call never reached the gateway rather than that one tool
forgot to log — and there is one place to hold the budget check, the error translation and the
trace id. A raw MCP server hands you the tool surface and leaves that endpoint design to you.
Every limit on this page is a counter in a store, and the store is resolved rather than assumed:
function isTcpRedisConfigured(): boolean {
const url = redisUrl();
return Boolean(url && (url.startsWith("redis://") || url.startsWith("rediss://")));
}
A URL test, two schemes, one boolean. redis:// and rediss:// are the TCP forms; anything else
— including the HTTP-based serverless Redis some deployments use instead — is not what this check
asks about. Keeping the test narrow lets the caller tell "no counter store is configured" apart
from "a counter store is configured in a shape this path does not speak".
The client-side consequence is why this matters on a page about gateways. Whatever the host is — a
VS Code extension, a desktop app, a CLI agent — the counter it is measured against lives on the
server. Two hosts calling the same key share one budget, limits survive a host restart, and
reinstalling the client does not grant a fresh allowance. Local enforcement cannot promise any of
that.
Resolution order matters when the store is missing, because it decides whether limits hold:
function resolveRedisMode(): RedisMode {
if (isUpstashRestConfigured()) return "upstash";
if (isTcpRedisConfigured()) return "tcp";
return "none";
}
Two checks in priority order and a constant: an Upstash-style REST configuration wins, then a TCP
URL, then none. The first two are live limiters; the third is the mode in which the limiter's
fail-open branch is the only branch that runs, and every call is admitted.
State that as a limitation rather than a feature. A remote MCP server with no counter store still
answers — availability first — but its per-key limits become advisory and the budget guard has
nothing to count against. The audit rows still appear, which is the useful part: a deployment that
degraded to none can be found afterwards from what the rows do not contain, instead of from a
spend surprise.
Each alternative is described the way its own documentation describes it; the links are sources,
not claims from this page.
Vendors describe this layer with different scopes, and the difference is instructive. Microsoft's
MCP Gateway project calls itself "a reverse proxy and management layer for MCP servers", aimed at
session-aware routing and lifecycle management
(Microsoft). Kong describes an MCP gateway as
centralising "access, security, and management for multiple Model Context Protocol servers"
(Kong). Both descriptions are
consistent with this page, and both emphasise the routing and lifecycle half — the half a
well-built proxy can reach. The half these excerpts show is per-caller accounting: which key
called, how often, at what spend, and the row that records it.
| What it governs | Where the limits live | What you pay | |
|---|---|---|---|
| A raw MCP server (stdio) | The tools it implements; nothing about the caller | Nowhere — the process trusts the host that launched it | Host machine time and your own engineering |
| An MCP server behind a plain reverse proxy | Forwarding, TLS, maybe an IP-level request cap | In the proxy configuration, blind to tool calls | The proxy tier, plus the metering you still build |
| A model-proxy AI gateway (Cloudflare ,Kong ) | Model traffic, per each vendor's own documentation | In the vendor's console or the proxy's config file | A subscription or a per-token fee; agent tool calls often stay outside its view |
| SmartGate — an MCP-native algorithm gateway | Which tools a key may call, how fast, how much it may spend, and the row that proves each call | In one policy object, read per request and clamped to the plan | Platform fee, plus a share only after measured savings pass the threshold |
The billing model is the part worth testing against your own invoices: pay for the platform, share only when you save. Free is $0 with 2M tokens a month, all seven tools, 120 MCP requests a
Does an MCP gateway replace the MCP server?
No. The server owns the tools — what they do, what arguments they take, what they return. The
gateway owns the caller: identity, rate, budget and the audit record. Removing the gateway leaves
a working server with no enforcement; removing the server leaves a gateway with nothing to route.
Is an MCP gateway the same thing as an MCP proxy?
A proxy moves bytes between a client and a server. A gateway parses enough of the request to
price it, attribute it and refuse it. Every gateway can proxy; the reverse is not true, because
per-key and per-month limits need fields a byte pipe never reads.
Where do the per-key limits actually apply?
In the gateway, against the key that authenticated the request, before the tool runs. Two hosts
sharing one key share one allowance, and the team ceiling is checked after the per-key window, so
one key cannot consume the whole team's budget in a burst.
What happens to limits if the counter store goes down?
The limiter fails open: calls are admitted with a flag naming the degraded path. That is a choice
for availability over enforcement, and it is visible in the response rather than buried in logs —
so a degraded window is something you can find afterwards.
Do budget caps need a code change when a plan changes?
No. The effective rate and daily cap are computed as the minimum of the team policy and the plan
capability, and the governance switches are forced off when the plan does not grant them. Editing
the plan catalog moves every team bound by it.
Which clients work without a local server process?
Any host that speaks Streamable HTTP over a URL. The gateway ships per-host configuration blocks
for the common ones, but the transport is the standard one — nothing here requires a process on
the caller's machine.
rest label.
The code in this article is not transcribed. Each block was cut directly out of the slice body
returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body
before publication; the first line inside every fence records the file and the exact source lines.
Symbols were pinned by whole-name containment (rule A level 2) and confirmed by the service's
slot-proof endpoint before any prose was written. Ten of the twelve excerpts are whole slice
bodies; the middleware and check_rate_limit_mcp blocks are windows into longer files, and the
lines around them are described rather than quoted.
Demand figures come from this project's own keyword run, recorded in research_brief.md and
search_volume.json: the section phrases ai gateway (2,400), llm gateway (1,600),
mcp proxy (720), mcp cli (480), ai agent security (480), openclaw mcp (480),
vscode mcp (390), remote mcp server (390), mcp router (260), model gateway (70),
api gateway for ai (20) and mcp firewall (20), each with its competition band; the head term
and its difficulty were measured in the same research pass.
| # | SERP keyword | Symbol | File | Source lines | How it was pinned | sha256(12) |
|---|---|---|---|---|---|---|
| 1 | mcp router | middleware |
middleware.ts |
72–89 | rule A L2 → slot-proof | 9ab198bee98b |
| 2 | api gateway for ai | infer_route |
backend/smartgate/core/audit_enrichment.py |
35–40 | rule A L2 → slot-proof | 8cb200979cc6 |
| 3 | mcp firewall | check_rate_limit_mcp |
backend/smartgate/core/rate_limiter.py |
51–77 | rule A L2 → slot-proof | b540f8154beb |
| 4 | ai gateway | clampGatewayPolicyToPlan |
lib/settings/clamp-gateway-policy.ts |
4–35 | rule A L2 → slot-proof | 613facc8545b |
| 5 | llm gateway | getPlanRateLimits |
lib/plan-rate-limits.ts |
5–12 | rule A L2 → slot-proof | 6d29a819fd91 |
| 6 | mcp proxy | loadGatewayPolicy |
lib/settings/load-gateway-policy.ts |
10–12 | rule A L2 → slot-proof | 82918a7047cc |
| 7 | model gateway | isEmptyPolicy |
lib/settings/plan-seeds.ts |
15–21 | rule A L2 → slot-proof | 142457b5325c |
| 8 | mcp cli | mcpConfigFilename |
lib/connect/mcp-config-templates.ts |
249–264 | rule A L2 → slot-proof | d60503ebfb68 |
| 9 | ai agent security | assertGatewayPolicyEditable |
lib/settings/assert-plan-allows.ts |
40–44 | rule A L2 → slot-proof | 5bf979730866 |
| 10 | openclaw mcp | smart_search |
backend/smartgate/api/mcp.py |
128–148 | rule A L2 → slot-proof | 12366ba0d241 |
| 11 | vscode mcp | isTcpRedisConfigured |
lib/redis/config.ts |
26–29 | rule A L2 → slot-proof | 7ca086a475d5 |
| 12 | remote mcp server | resolveRedisMode |
lib/redis/config.ts |
38–42 | rule A L2 → slot-proof | 3a9dd8e9daf4 |
Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before
publication. 12 of 12 sections pinned, 0 abstentions, 0 misses.