# Introducing Swarm: Multi-Agent Orchestration and an LLM Gateway in Pure Rust 🦀

> Source: <https://dev.to/fcn06/introducing-swarm-multi-agent-orchestration-and-an-llm-gateway-in-pure-rust-54ha>
> Published: 2026-08-18 05:48:26+00:00

While experimenting with multi-agent systems, I kept ending up with two separate pieces of infrastructure: an orchestration layer for agents and tools, and a gateway layer for routing LLM requests.

I wanted both to share the same runtime, provider abstractions, state management, and protocol contracts.

So I built ** Swarm**, an open-source AI orchestration framework and model gateway written in Rust.

Many AI stacks end up separating these concerns: a dedicated proxy for lightweight routing and a separate orchestrator for more complex reasoning. Swarm unifies both patterns around a single high-performance Tokio runtime.

```
+--------------------------------------------------------------------------------------------------+
|                                        SWARM MODES                                               |
+--------------------------------------------------------------------------------------------------+
|                                                                                                  |
|   MODE 1: MULTI-AGENT & MCP ORCHESTRATION               MODE 2: MODEL GATEWAY SERVER             |
|   (kickstart/multi_agent_orchestration_kickstart/)      (kickstart/gateway_kickstart/)           |
|                                                                                                  |
|   • Planner Agent (Dynamic plan generation)             • POST /v1/chat/completions (OpenAI)     |
|   • Executor Agent (Workflow DAG execution)             • POST /v1/responses (Open Responses)    |
|   • Domain Specialists with MCP Tool integration        • Stateful multi-turn chaining          |
|   • Discovery & Memory services                         • Multi-provider (Groq, Gemini, OpenAI,  |
|   • Evaluation & Judge Service                            Ollama / vLLM / local endpoints)       |
|   • Resilient OAuth2 / JWT authentication               • High-throughput lock-free cache        |
|                                                                                                  |
+--------------------------------------------------------------------------------------------------+
```

The key idea: Swarm can run as a full agent orchestration stack or as a standalone LLM gateway without requiring two unrelated frameworks.

Coordinating multiple agents becomes much easier when service boundaries and message contracts are explicit.

Mode 1 splits responsibilities across decoupled, specialized services:

Inter-agent communication relies on type-safe agent-to-agent (A2A) message contracts, catching many contract and integration errors during development and compilation.

```
User Request
    ↓
Planner
    ↓
Execution DAG
    ↓
Executor
    ↓
Weather Agent
    ↓
MCP Weather Tool
    ↓
Evaluation
    ↓
Final Response
```

Mode 2 exposes an OpenAI-compatible gateway for client applications, developer tools, and automated pipelines.

`POST /v1/chat/completions`

):`POST /v1/responses`

):`previous_response_id`

references.

```
[server]
bind_address = "0.0.0.0:8080"
log_level = "info"

[models]
default_model = "openai/gpt-oss-20b"

[providers.groq]
api_url = "https://api.groq.com/openai/v1/chat/completions"

[providers.google]
api_url = "https://generativelanguage.googleapis.com/v1beta/models"

[providers.custom]
# Local inference (Ollama / vLLM / llama.cpp / LocalAI)
api_url = "http://localhost:11434/v1/chat/completions"
recommended_models = ["llama3.2:latest", "mistral:latest", "deepseek-r1:8b"]
```

Rust gives Swarm a few useful properties for orchestration and gateway workloads:

`DashMap`

and `Arc`

-based stores allow shared state across concurrent requests without a global application lock.You can test either mode locally in minutes:

```
git clone https://github.com/fcn06/swarm.git
cd swarm

# Configure your provider keys
cp .env.example .env
./kickstart/gateway_kickstart/01_launch_gateway.sh

# Test OpenAI-compatible completions
curl -X POST http://localhost:8080/v1/chat/completions   -H "Content-Type: application/json"   -d '{
    "model": "openai/gpt-oss-20b",
    "messages": [{"role": "user", "content": "Explain Swarm architecture in 2 sentences."}]
  }'
# Launch agents, MCP server, and infrastructure services
./kickstart/multi_agent_orchestration_kickstart/01_launch_all.sh

# Run a live MCP tool test query
./kickstart/multi_agent_orchestration_kickstart/02_test_weather_query.sh "What is the current weather in Boston ?"
```

Swarm is fully open-source under the Apache-2.0 license. We rely on and contribute back to the emerging Rust AI ecosystem, including the official [MCP Rust SDK](https://github.com/modelcontextprotocol/rust-sdk) and [A2A Protocol](https://github.com/EmilLindfors/a2a-rs).

I'm especially interested in feedback from people running agent systems or LLM gateways in production:

**Would you rather deploy the orchestration and gateway as one runtime, or keep them completely separate?**

If you try Swarm, I'd also love feedback on the MCP runtime, gateway compatibility, and APIs.