# You Don't Need to Choose Between a Gateway and an Agent Framework > Source: > Published: 2026-08-21 15:32:51+00:00 When I first published [Swarm](https://github.com/fcn06/swarm) on GitHub, most questions weren't about Rust or MCP. They were about timing and categorization: "We just need a lightweight gateway for multi-provider routing; agents feel like overkill." "We already run an orchestration framework; why would we replace our proxy?" This reaction highlights a false dichotomy currently plaguing the AI infrastructure ecosystem: the assumption that a gateway and an agent orchestrator must be two completely different products. In practice, teams rarely wake up needing full-blown multi-agent autonomous swarms on Day 1. But when they start with a standalone proxy, they inevitably hit a wall — patching together Python microservices, external vector state stores, MCP bridges, and ad-hoc eval scripts. Every evolution requires a rewrite. The core premise of Swarm is different: a single, pure-Rust runtime where you don't choose between a gateway and an orchestrator — you simply choose which capabilities to turn on. Most engineering teams evolve their LLM stack along a predictable trajectory: ``` Rung 1: OpenAI-Compatible Gateway (Drop-in replacement for hardcoded SDKs) └── Rung 2: Multi-Provider Fallbacks (Groq, Gemini, Ollama, vLLM via TOML) └── Rung 3: Stateful Sessions (Previous response chaining & context) └── Rung 4: Native MCP Tools (SSE + Streamable HTTP tool execution) └── Rung 5: Multi-Agent DAGs (Planner + Executor + Specialists) └── Rung 6: Built-in Evals (LLM-as-a-Judge & policy gates) ``` You can stop at any rung and have a lean, production-grade binary. When you're ready for the next level, you change a configuration flag — not your architectural foundation. If your immediate goal is simply eliminating hardcoded API keys and single-vendor SDK locks, Swarm acts as an OpenAI-compatible drop-in front door with sub-millisecond native routing overhead. ``` # Spin up the gateway in seconds ./kickstart/gateway_kickstart/01_launch_gateway.sh curl -X POST http://localhost:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Explain progressive disclosure in software."}] }' ``` You get instant OpenAI compatibility. No agent overhead, no background worker queues, no forced abstractions. When rate limits hit or you need cost-effective fallbacks across cloud and local runtimes (Groq, Anthropic, Gemini, Ollama, vLLM, llama.cpp), routing is declared cleanly in `config.toml` : ``` [providers.groq] api_url = "https://api.groq.com/openai/v1/chat/completions" weight = 80 [providers.local_vllm] api_url = "http://localhost:8000/v1/chat/completions" recommended_models = ["meta-llama/Llama-3.3-70B-Instruct"] ``` Your applications continue calling the same `/v1/chat/completions` endpoint. Failover, load distribution, and local-inference routing happen invisibly inside the runtime. `/v1/responses` Multi-turn chat state is where teams often bolt on an external Redis or PostgreSQL session manager. Swarm provides explicit turn-by-turn state management natively through `/v1/responses` using `previous_response_id` chaining: ``` curl -X POST http://localhost:8080/v1/responses \ -H "Content-Type: application/json" \ -d '{ "model": "groq/llama-3.3-70b-versatile", "input": "Calculate the Q3 cloud infrastructure spend.", "previous_response_id": "resp_01JMW892KPA7XYZ" }' ``` State is managed by the runtime, eliminating client-side conversation bloat while keeping state inspection simple and deterministic. When your model needs real-world context — database schemas, filesystem access, API calls — you shouldn't have to migrate to a heavy agent framework just to call tools. Swarm natively supports MCP (over both SSE and streamable HTTP) directly inside the gateway layer: ``` [mcp_servers.postgres_db] transport = "sse" url = "http://localhost:3001/sse" [mcp_servers.git_tools] transport = "http" url = "http://localhost:3002/mcp" ``` Tool discovery, argument validation, and streaming tool execution run within the same engine that routes your completions. When single-prompt loops cannot solve compound tasks, Swarm activates its autonomous orchestration engine: ``` User Intent │ ▼ [ Planner ] ──► Builds Execution DAG (Dependencies & Concurrency) │ ▼ [ Executor ] ──► Dispatches tasks across Domain Specialists │ ├── Specialist A (Data Analyst + Postgres MCP) └── Specialist B (Report Writer + File MCP) │ ▼ Unified Response ``` **Why this matters:** Rung 5 reuses the identical provider configurations, fallback pools, state engine, and MCP tool connectors established in Rungs 1–4. There is no secondary agent daemon or translation bridge. The final rung is the one most gateways and agent frameworks omit entirely: closing the loop on quality. Instead of exporting logs to an external SaaS pipeline, Swarm embeds an LLM-as-a-Judge loop. It scores intermediate DAG outputs, validates MCP tool results against deterministic schemas, and flags hallucinated responses before they reach client applications: ``` curl -X POST http://localhost:8080/v1/eval/judge \ -H "Content-Type: application/json" \ -d '{ "response_id": "resp_01JMW892KPA7XYZ", "criteria": ["correctness", "grounding", "conciseness"], "judge_model": "openai/gpt-4o" }' ``` This foundational layer enables our upcoming roadmap items: policy-based dynamic routing, durable state checkpoints, and human-in-the-loop validation gates. The individual capabilities of Swarm — gateway proxying, MCP tool invocation, DAG planning, automated evaluation — exist across different open-source projects. What is rare is finding them integrated into a single, zero-dependency, memory-safe binary where adopting multi-agent orchestration doesn't invalidate the proxy architecture you set up on Day 1. The architectural bet of Swarm is simple: **the tools you choose when you only need a gateway should never become technical debt the day you need agents.** If you're currently scaling your LLM infrastructure: Check out the project and try the kickstart scripts on GitHub: [github.com/fcn06/swarm](https://github.com/fcn06/swarm)