Introducing Swarm: Multi-Agent Orchestration and an LLM Gateway in Pure Rust 🦀

A developer has open-sourced Swarm, a Rust-based AI orchestration framework and model gateway that unifies multi-agent orchestration and LLM routing in a single Tokio runtime. The framework supports two modes: a multi-agent orchestration stack with planner, executor, and MCP tool integration, and an OpenAI-compatible gateway server with multi-provider support. Built in Rust, Swarm leverages DashMap and Arc for concurrent state management and type-safe A2A message contracts.

While experimenting with multi-agent systems, I kept ending up with two separate pieces of infrastructure: an orchestration layer for agents and tools, and a gateway layer for routing LLM requests. I wanted both to share the same runtime, provider abstractions, state management, and protocol contracts. So I built Swarm , an open-source AI orchestration framework and model gateway written in Rust. Many AI stacks end up separating these concerns: a dedicated proxy for lightweight routing and a separate orchestrator for more complex reasoning. Swarm unifies both patterns around a single high-performance Tokio runtime. +--------------------------------------------------------------------------------------------------+ | SWARM MODES | +--------------------------------------------------------------------------------------------------+ | | | MODE 1: MULTI-AGENT & MCP ORCHESTRATION MODE 2: MODEL GATEWAY SERVER | | kickstart/multi agent orchestration kickstart/ kickstart/gateway kickstart/ | | | | • Planner Agent Dynamic plan generation • POST /v1/chat/completions OpenAI | | • Executor Agent Workflow DAG execution • POST /v1/responses Open Responses | | • Domain Specialists with MCP Tool integration • Stateful multi-turn chaining | | • Discovery & Memory services • Multi-provider Groq, Gemini, OpenAI, | | • Evaluation & Judge Service Ollama / vLLM / local endpoints | | • Resilient OAuth2 / JWT authentication • High-throughput lock-free cache | | | +--------------------------------------------------------------------------------------------------+ The key idea: Swarm can run as a full agent orchestration stack or as a standalone LLM gateway without requiring two unrelated frameworks. Coordinating multiple agents becomes much easier when service boundaries and message contracts are explicit. Mode 1 splits responsibilities across decoupled, specialized services: Inter-agent communication relies on type-safe agent-to-agent A2A message contracts, catching many contract and integration errors during development and compilation. User Request ↓ Planner ↓ Execution DAG ↓ Executor ↓ Weather Agent ↓ MCP Weather Tool ↓ Evaluation ↓ Final Response Mode 2 exposes an OpenAI-compatible gateway for client applications, developer tools, and automated pipelines. POST /v1/chat/completions : POST /v1/responses : previous response id references. server bind address = "0.0.0.0:8080" log level = "info" models default model = "openai/gpt-oss-20b" providers.groq api url = "https://api.groq.com/openai/v1/chat/completions" providers.google api url = "https://generativelanguage.googleapis.com/v1beta/models" providers.custom Local inference Ollama / vLLM / llama.cpp / LocalAI api url = "http://localhost:11434/v1/chat/completions" recommended models = "llama3.2:latest", "mistral:latest", "deepseek-r1:8b" Rust gives Swarm a few useful properties for orchestration and gateway workloads: DashMap and Arc -based stores allow shared state across concurrent requests without a global application lock.You can test either mode locally in minutes: git clone https://github.com/fcn06/swarm.git cd swarm Configure your provider keys cp .env.example .env ./kickstart/gateway kickstart/01 launch gateway.sh Test OpenAI-compatible completions curl -X POST http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{ "model": "openai/gpt-oss-20b", "messages": {"role": "user", "content": "Explain Swarm architecture in 2 sentences."} }' Launch agents, MCP server, and infrastructure services ./kickstart/multi agent orchestration kickstart/01 launch all.sh Run a live MCP tool test query ./kickstart/multi agent orchestration kickstart/02 test weather query.sh "What is the current weather in Boston ?" Swarm is fully open-source under the Apache-2.0 license. We rely on and contribute back to the emerging Rust AI ecosystem, including the official MCP Rust SDK https://github.com/modelcontextprotocol/rust-sdk and A2A Protocol https://github.com/EmilLindfors/a2a-rs . I'm especially interested in feedback from people running agent systems or LLM gateways in production: Would you rather deploy the orchestration and gateway as one runtime, or keep them completely separate? If you try Swarm, I'd also love feedback on the MCP runtime, gateway compatibility, and APIs.