FastMCP Agent Mail: RBAC Tokens vs Anonymous Access, and the 403 Errors in Between

A developer detailed how FastMCP servers default to anonymous access, which becomes a security risk when exposed via HTTP behind an ingress, leading to confusing 403 errors. The post explains how to diagnose 401 vs 403 errors and recommends using FastMCP's StaticTokenVerifier for internal agent mail servers.

A FastMCP server started with fastmcp run server.py accepts every request from every client, because the default configuration ships with no authentication at all. That's fine on localhost. Then you move the same server behind a Kubernetes IngressRoute with TLS, point three different agent sessions at it, and suddenly the agent reports that its mail tools "aren't available" while the pod logs show a stream of 403 Forbidden . Nothing about the server changed. The environment did, and anonymous access stopped being an option the moment the endpoint became reachable by anything other than you. This one's for anyone running an agent coordination server agent mail, shared memory, task queues as an MCP service that multiple Claude Code or Codex sessions talk to over HTTP. The pattern generalizes: any FastMCP server that graduates from stdio-on-localhost to streamable HTTP behind an ingress hits the same wall, and the failure mode is quieter than you'd expect. When you develop an MCP server locally, you're usually on stdio transport. The client spawns the server process directly, so "authentication" is just filesystem permissions. There's no network boundary, no tokens, nothing to get wrong. This is why local development feels so smooth and why it teaches you nothing about production. Switch to streamable HTTP which you need for multiple agents sharing one server and the situation inverts. Now anyone who can reach the port can call initialize , list your tools, and invoke them. For an agent mail server, that means reading every message between your agents and injecting new ones. If your agents treat inbound mail as instructions, and coordination servers exist precisely so agents act on each other's messages, an unauthenticated mail endpoint is a prompt injection channel with a REST API. FastMCP 2.x makes auth opt-in via the auth parameter on the server constructor. If you don't pass one, you get anonymous access. The docs are clear about this, but the gap between "docs are clear" and "you actually did it before exposing the ingress" is where the trouble lives. Here's the diagnostic detail that saves you an hour: a 403 on an MCP endpoint can originate from three different layers, and they look almost identical from the client side. 403 Forbidden text, and the pod logs show nothing. 401 with a WWW-Authenticate header. A 403 .The 401-vs-403 distinction matters more than it seems. A 401 means "I don't know who you are": your header is missing, malformed, or the token doesn't verify. A 403 means "I know who you are and the answer is no": the token parsed fine but lacks a required scope. When you're staring at agent logs at the end of a long debugging session, that one digit tells you whether to check the client config 401 or the server's scope requirements 403 . The reason this gets miserable is the client side. Claude Code doesn't surface the HTTP status prominently. The server just shows as failed in /mcp , the tools vanish from the agent's toolset, and the agent either tells you the capability doesn't exist or, worse, improvises around it. A running pod, a green health check, and a completely non-functional toolset can coexist happily. If you take one thing from this post, it's that "the pod is Running" verifies nothing about whether an agent can call a single tool. For an internal agent mail server, you don't need a full OAuth flow. FastMCP 2.12 ships a StaticTokenVerifier that maps opaque token strings to identities and scopes, which is exactly the right weight for a homelab or internal deployment: python import os from fastmcp import FastMCP from fastmcp.server.auth.providers.jwt import StaticTokenVerifier verifier = StaticTokenVerifier tokens={ os.environ "MAIL TOKEN WORKER" : { "client id": "agent-worker", "scopes": "mail:read", "mail:write" , }, os.environ "MAIL TOKEN REVIEWER" : { "client id": "agent-reviewer", "scopes": "mail:read" , read-only: can fetch inbox, can't send }, }, required scopes= "mail:read" , mcp = FastMCP "agent-mail", auth=verifier Two things to notice. The token values come from environment variables, never literals in the file, so the tokens live in a Kubernetes Secret and get injected at pod start. And each agent role gets its own token with its own scopes. That second part is the actual RBAC: a reviewer agent that only triages messages has no business holding a credential that can send them. This is the same two-tier thinking I wrote about in agent credential management https://guatulabs.dev/posts/agent-credential-management-two-tier-service-accounts/ , applied one layer down at the MCP transport. Per-tool enforcement then reads the validated token from the request context: python from fastmcp.server.dependencies import get access token from fastmcp.exceptions import ToolError @mcp.tool async def send message to: str, subject: str, body: str - dict: token = get access token if "mail:write" not in token.scopes: raise ToolError "This credential is read-only." return await deliver to, subject, body If you outgrow static tokens more than a handful of agents, or tokens that need rotation without a redeploy , swap StaticTokenVerifier for FastMCP's JWTVerifier pointed at a JWKS endpoint. The server code barely changes; the constructor argument does. I covered the boilerplate side of FastMCP in an earlier post https://guatulabs.dev/posts/building-mcp-servers-with-fastmcp/ ; auth is the part that post's happy path skipped. The broken client configs I see fall into two buckets. The first is a stale stdio entry pointing at a script that moved or was replaced by the HTTP deployment: { "mcpServers": { "agent-mail": { "command": "python", "args": "/home/user/old-scripts/mail server.py" } } } This fails instantly and at least fails loudly. The second bucket is the sneaky one: the URL was updated to the new HTTPS endpoint but the Authorization header never got added, because the server didn't require one when the config was written. That's the config that worked for weeks and then started returning 403 the day auth landed on the server. The corrected version, with the token pulled from the environment rather than committed in plaintext: { "mcpServers": { "agent-mail": { "type": "http", "url": "https://mail.example.com/mcp", "headers": { "Authorization": "Bearer ${AGENT MAIL TOKEN}" } } } } Claude Code expands ${VAR} references in .mcp.json from the environment. Use that. A raw token string in a JSON file in your home directory has a way of ending up in dotfile repos, pair-debugging screenshots, and pasted "here's my config, what's wrong" messages. The env-var indirection costs you one line in a shell profile and removes an entire category of leak. Nothing exotic on the Kubernetes side. Traefik terminates TLS and forwards to the service; FastMCP handles auth itself, so no ForwardAuth middleware is needed: apiVersion: traefik.io/v1alpha1 kind: IngressRoute metadata: name: agent-mail namespace: agents spec: entryPoints: - websecure routes: - match: Host mail.example.com && PathPrefix /mcp kind: Rule services: - name: agent-mail port: 8000 tls: secretName: mail-example-com-tls Keeping auth in the application rather than the ingress is a deliberate choice here. The server needs to know which agent is calling to enforce scopes anyway, so putting a second auth layer in Traefik just gives you two places to misconfigure and two flavors of 403 to tell apart. If you already run ForwardAuth everywhere as policy, fine, but then remember that layer exists when you debug. Before touching any agent config, prove the server behaves correctly with curl. First the failure case, no token: curl -i https://mail.example.com/mcp \ -H "Accept: application/json, text/event-stream" \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize", "params":{"protocolVersion":"2025-06-18","capabilities":{}, "clientInfo":{"name":"curl","version":"0"}}}' You want HTTP/2 401 with a WWW-Authenticate: Bearer header. If you get Traefik's plain 403 or a 404, the request never reached FastMCP and your problem is routing, not auth. Then the success case: curl -i https://mail.example.com/mcp \ -H "Authorization: Bearer $AGENT MAIL TOKEN" \ -H "Accept: application/json, text/event-stream" \ -H "Content-Type: application/json" \ -d '{"jsonrpc":"2.0","id":1,"method":"initialize", ...}' A 200 with an initialize result means the full path works: DNS, TLS, ingress, pod, auth. Only now is a client-side failure actually a client-side failure. This two-curl check takes thirty seconds and cleanly bisects the problem, which beats restarting agent sessions and squinting at /mcp output. The silent failure is the real enemy. An agent whose MCP server fails auth doesn't crash. It just proceeds without those tools. In a multi-agent setup where sessions coordinate through mail, one agent silently losing its mailbox looks like that agent "deciding" not to communicate. Check /mcp status at session start, or better, make your agents' startup routine fetch their inbox once and treat failure as fatal rather than shrugging past it. Health endpoints lie by omission. If you expose a custom unauthenticated /health route for Kubernetes probes, understand what you've built: a check that confirms the process is up while saying nothing about whether authenticated tool calls succeed. Reasonable for liveness. Useless for "are the agents actually able to use this." Valid token, wrong scope, confusing error. The read-only reviewer token from the example above will initialize successfully and list tools, then fail on send message . From the agent's perspective the tool exists but errors out. Make the tool-level error message state the actual problem "this credential is read-only" , because the agent will relay that message to you verbatim, and "permission denied" tells you nothing. Documentation drift bites here too. If your CLAUDE.md or agent instructions enumerate the mail server's tools and you later add auth-gated ones, agents will attempt calls their token can't make. Keep the documented capability list synchronized with what each role can actually invoke, not with what the server exposes in total. It's the same lesson as semantic index drift https://guatulabs.dev/posts/silent-drift-why-re-embedding-only-on-count-changes-rots-your-semantic-index/ : any description of a system that isn't regenerated from the system will eventually be wrong. An alternative I considered and rejected: mTLS between agents and the server. It authenticates the machine, not the agent role, and distributing client certs to ephemeral agent sessions is far more friction than handing each role a scoped bearer token. Certificates make sense when the caller is a long-lived service. Agent sessions aren't. The rule I'd apply: the moment an MCP server leaves stdio, it gets a token verifier, even if the only network it's exposed on is your own. Not because your LAN is hostile, but because the anonymous configuration silently becomes load-bearing, and you'll forget it's there until the day you add an ingress, a Tailscale route, or a second user. Retrofitting auth after three agents and two config files depend on anonymous access is strictly worse than starting with a static token that takes ten lines. For a mail server specifically, the stakes are higher than for a read-only lookup tool. Messages are instructions. Scoping who can write them is the difference between a coordination layer and an attack surface. If you're building out multi-agent infrastructure and want a second pair of eyes on the security model, that's work I consult on https://guatulabs.com/services . And when the 403 does show up: read the status code, run the two curls, and find out which layer is saying no before you change anything. Most of the pain in these debugging sessions comes from fixing the wrong layer first.