{"slug": "introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust", "title": "Introducing Swarm: Multi-Agent Orchestration and an LLM Gateway in Pure Rust 🦀", "summary": "A developer has open-sourced Swarm, a Rust-based AI orchestration framework and model gateway that unifies multi-agent orchestration and LLM routing in a single Tokio runtime. The framework supports two modes: a multi-agent orchestration stack with planner, executor, and MCP tool integration, and an OpenAI-compatible gateway server with multi-provider support. Built in Rust, Swarm leverages DashMap and Arc for concurrent state management and type-safe A2A message contracts.", "body_md": "While experimenting with multi-agent systems, I kept ending up with two separate pieces of infrastructure: an orchestration layer for agents and tools, and a gateway layer for routing LLM requests.\n\nI wanted both to share the same runtime, provider abstractions, state management, and protocol contracts.\n\nSo I built ** Swarm**, an open-source AI orchestration framework and model gateway written in Rust.\n\nMany AI stacks end up separating these concerns: a dedicated proxy for lightweight routing and a separate orchestrator for more complex reasoning. Swarm unifies both patterns around a single high-performance Tokio runtime.\n\n```\n+--------------------------------------------------------------------------------------------------+\n|                                        SWARM MODES                                               |\n+--------------------------------------------------------------------------------------------------+\n|                                                                                                  |\n|   MODE 1: MULTI-AGENT & MCP ORCHESTRATION               MODE 2: MODEL GATEWAY SERVER             |\n|   (kickstart/multi_agent_orchestration_kickstart/)      (kickstart/gateway_kickstart/)           |\n|                                                                                                  |\n|   • Planner Agent (Dynamic plan generation)             • POST /v1/chat/completions (OpenAI)     |\n|   • Executor Agent (Workflow DAG execution)             • POST /v1/responses (Open Responses)    |\n|   • Domain Specialists with MCP Tool integration        • Stateful multi-turn chaining          |\n|   • Discovery & Memory services                         • Multi-provider (Groq, Gemini, OpenAI,  |\n|   • Evaluation & Judge Service                            Ollama / vLLM / local endpoints)       |\n|   • Resilient OAuth2 / JWT authentication               • High-throughput lock-free cache        |\n|                                                                                                  |\n+--------------------------------------------------------------------------------------------------+\n```\n\nThe key idea: Swarm can run as a full agent orchestration stack or as a standalone LLM gateway without requiring two unrelated frameworks.\n\nCoordinating multiple agents becomes much easier when service boundaries and message contracts are explicit.\n\nMode 1 splits responsibilities across decoupled, specialized services:\n\nInter-agent communication relies on type-safe agent-to-agent (A2A) message contracts, catching many contract and integration errors during development and compilation.\n\n```\nUser Request\n    ↓\nPlanner\n    ↓\nExecution DAG\n    ↓\nExecutor\n    ↓\nWeather Agent\n    ↓\nMCP Weather Tool\n    ↓\nEvaluation\n    ↓\nFinal Response\n```\n\nMode 2 exposes an OpenAI-compatible gateway for client applications, developer tools, and automated pipelines.\n\n`POST /v1/chat/completions`\n\n):`POST /v1/responses`\n\n):`previous_response_id`\n\nreferences.\n\n```\n[server]\nbind_address = \"0.0.0.0:8080\"\nlog_level = \"info\"\n\n[models]\ndefault_model = \"openai/gpt-oss-20b\"\n\n[providers.groq]\napi_url = \"https://api.groq.com/openai/v1/chat/completions\"\n\n[providers.google]\napi_url = \"https://generativelanguage.googleapis.com/v1beta/models\"\n\n[providers.custom]\n# Local inference (Ollama / vLLM / llama.cpp / LocalAI)\napi_url = \"http://localhost:11434/v1/chat/completions\"\nrecommended_models = [\"llama3.2:latest\", \"mistral:latest\", \"deepseek-r1:8b\"]\n```\n\nRust gives Swarm a few useful properties for orchestration and gateway workloads:\n\n`DashMap`\n\nand `Arc`\n\n-based stores allow shared state across concurrent requests without a global application lock.You can test either mode locally in minutes:\n\n```\ngit clone https://github.com/fcn06/swarm.git\ncd swarm\n\n# Configure your provider keys\ncp .env.example .env\n./kickstart/gateway_kickstart/01_launch_gateway.sh\n\n# Test OpenAI-compatible completions\ncurl -X POST http://localhost:8080/v1/chat/completions   -H \"Content-Type: application/json\"   -d '{\n    \"model\": \"openai/gpt-oss-20b\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Explain Swarm architecture in 2 sentences.\"}]\n  }'\n# Launch agents, MCP server, and infrastructure services\n./kickstart/multi_agent_orchestration_kickstart/01_launch_all.sh\n\n# Run a live MCP tool test query\n./kickstart/multi_agent_orchestration_kickstart/02_test_weather_query.sh \"What is the current weather in Boston ?\"\n```\n\nSwarm is fully open-source under the Apache-2.0 license. We rely on and contribute back to the emerging Rust AI ecosystem, including the official [MCP Rust SDK](https://github.com/modelcontextprotocol/rust-sdk) and [A2A Protocol](https://github.com/EmilLindfors/a2a-rs).\n\nI'm especially interested in feedback from people running agent systems or LLM gateways in production:\n\n**Would you rather deploy the orchestration and gateway as one runtime, or keep them completely separate?**\n\nIf you try Swarm, I'd also love feedback on the MCP runtime, gateway compatibility, and APIs.", "url": "https://wpnews.pro/news/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust", "canonical_source": "https://dev.to/fcn06/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust-54ha", "published_at": "2026-08-18 05:48:26+00:00", "updated_at": "2026-08-18 06:11:56.589500+00:00", "lang": "en", "topics": ["ai-agents", "ai-infrastructure", "developer-tools", "artificial-intelligence"], "entities": ["Swarm", "Rust", "Tokio", "MCP", "OpenAI", "Groq", "Gemini", "Ollama"], "alternates": {"html": "https://wpnews.pro/news/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust", "markdown": "https://wpnews.pro/news/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust.md", "text": "https://wpnews.pro/news/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust.txt", "jsonld": "https://wpnews.pro/news/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust.jsonld"}}