A conversation that carries forward.
Keep item history and response state on the server. Continue with previous_response_id, without rebuilding the transcript on every call.
→ response_02
→ response_03
vLLM Agentic API is the agentic server that lets you run Codex and Claude Code on top of vLLM.
Everything you need for agentic inference: conversation state, built-in tools, and multi-turn execution with the open models you choose.
Agentic inference brings model calls, built-in tool execution, and conversation state together. vLLM serves the model; Agentic API coordinates the loop.
Execute built-in web search and tools from Model Context Protocol (MCP) servers on the gateway. Tool call outputs flow back to the model so it can take the next step.
Let the gateway coordinate model calls and built-in tools until the response is ready or the client needs to act.
REASON → ACT → CONTINUE Use HTTP for requests, server-sent events (SSE) for streamed responses, and WebSockets for interactive Responses API clients.
Connect to compatible open models served by vLLM. Choose the model and tool-calling configuration for your workload.
Clear execution boundaries. Codex and Claude Code still execute their own shell and editor tools. Agentic API runs the tools assigned to the gateway.
Built for vLLM, with recorded integration tests for SGLang and NVIDIA Dynamo covering streaming, multi-turn state, and client-executed function calls.
Connect Agentic API to SGLang’s Responses endpoint. The gateway stores response history and carries it into the next turn.
Point Agentic API at the Dynamo frontend to add conversation state and continuation to your distributed inference stack.
Point your coding agent at Agentic API. Keep your familiar workflow, with an open model behind it.
Run a tool-capable model with vLLM. The example uses an upstream on port 5050.
Run v0.9.0 from PyPI with uvx or install from crates.io, with source builds available for development.
Install Codex or Claude Code, then choose your client and served model.
cargo install agentic-server --version 0.9.0 --locked
Install the released gateway and agentic CLI. Requires Rust and Cargo.
Run either command to start Agentic API and launch the harness:
agentic run codex \
--upstream http://127.0.0.1:5050 \
--model Qwen/Qwen3-30B-A3B-FP8
agentic run claude \
--upstream http://127.0.0.1:5050 \
--model Qwen/Qwen3-30B-A3B-FP8
For the desktop app, see the Codex Desktop setup guide.
Meet the people building the application layer for open-model agents. There’s room for your ideas, your fixes, and your next contribution.
Join us in #sig-agentic-api on the vLLM Slack.