aisuite is Andrew Ng's MIT-licensed Python library for building with LLMs, giving you one OpenAI-style Chat Completions API across OpenAI, Anthropic, Google, Ollama, and more, plus a first-class Agents API with tools, toolkits, policies, and native MCP support — and the engine behind the OpenWorker desktop app.
- ⭐ 15876
- Python
- MIT
- Updated 2026-08-02
LiteLLM — Unified OpenAI-Compatible API for 100+ LLM Providers • LLM Gateway: Portkey vs LiteLLM vs OpenRouter
What Is aisuite? # #
aisuite is a lightweight, MIT-licensed Python library from Andrew Ng’s team for building with LLMs, structured in two layers: a unified Chat Completions API across providers, and an Agents API with tools, toolkits, and MCP support on top of it. It’s also the engine behind OpenWorker, a separate desktop AI-coworker app now developed in its own repository — the aisuite README still carries a snapshot of OpenWorker’s old in-repo source under platform/
, but active OpenWorker development has moved out.
🔗 GitHub: https://github.com/andrewyng/aisuite 📦 PyPI: https://pypi.org/project/aisuite
At 15,800+ GitHub stars, MIT licensed, with a commit as recent as July 25, 2026, and carrying Andrew Ng’s name (Coursera/deeplearning.ai co-founder, one of the most recognized figures in applied ML education), it’s a credible, actively maintained entry in an increasingly crowded “one API for every LLM provider” category.
The project’s own architecture diagram, reproduced from the README:
┌───────────────────────────────────────────────┐
│ OpenWorker (separate repo) │ agent harness for doing everyday tasks
├───────────────────────────────────────────────┤
│ Agents API · Toolkits · MCP │ build agents across multiple LLMs
├───────────────────────────────────────────────┤
│ Chat Completions API │ one API across multiple LLM providers
├────────┬───────────┬────────┬────────┬────────┤
│ OpenAI │ Anthropic │ Google │ Ollama │ Others │
└────────┴───────────┴────────┴────────┴────────┘
Installation # #
pip install aisuite # base package, no provider SDKs
pip install 'aisuite[anthropic]' # with a specific provider's SDK
pip install 'aisuite[all]' # with all provider SDKs
You’ll still need your own API keys for whichever providers you call — aisuite doesn’t proxy billing or provide free access, it just standardizes the interface.
Chat Completions: One API Across Providers # #
The Chat Completions layer is a high-level abstraction over model calls — it supports the common parameters (temperature
, max_tokens
, tools
, etc.) in a provider-agnostic way and normalizes request/response shapes so provider-specific SDK differences don’t leak into your application code.
Models are addressed as <provider>:<model-name>
, and aisuite routes each call to the right provider with the right parameters:
import aisuite as ai
client = ai.Client()
models = ["openai:gpt-4o", "anthropic:claude-3-5-sonnet-20240620"]
messages = [
{"role": "system", "content": "Respond in Pirate English."},
{"role": "user", "content": "Tell me a joke."},
]
for model in models:
response = client.chat.completions.create(
model=model,
messages=messages,
temperature=0.75
)
print(response.choices[0].message.content)
Streaming #
for chunk in client.chat.completions.create(model=model, messages=messages, stream=True):
print(chunk.choices[0].delta.content or "", end="", flush=True)
Streaming works across OpenAI, Anthropic, Ollama, and OpenAI-compatible endpoints, with an async variant (await client.chat.completions.acreate(...)
, iterated with async for
). Tool calls stream too, as incremental delta.tool_calls
fragments — but note that streamed tool calling is manual only; it can’t be combined with the automatic max_turns
loop described below.
Agents API: Tools, Toolkits, and Policies # #
aisuite turns tool calling into passing plain Python functions — it generates the schema, executes the call, and feeds the result back to the model for you.
Automatic tool-call loop with max_turns
#
def will_it_rain(location: str, time_of_day: str):
"""Check if it will rain in a location at a given time today.
Args:
location (str): Name of the city
time_of_day (str): Time of the day in HH:MM format.
"""
return "YES"
client = ai.Client()
response = client.chat.completions.create(
model="openai:gpt-4o",
messages=[{
"role": "user",
"content": "I live in San Francisco. Can you check for weather "
"and plan an outdoor picnic for me at 2pm?"
}],
tools=[will_it_rain],
max_turns=2 # Maximum number of back-and-forth tool calls
)
print(response.choices[0].message.content)
With max_turns
set, aisuite sends the message, executes any tool calls the model requests, feeds results back, and repeats until the conversation completes — response.choices[0].intermediate_messages
carries the full tool-interaction history. Omit max_turns
for full manual control: aisuite then just returns the model’s tool-call requests and you drive the loop yourself.
The structured Agents API #
For longer-running, multi-step work, aisuite has a first-class Agent
/ Runner
API with prebuilt toolkits:
import aisuite as ai
from aisuite import Agent, Runner
agent = Agent(
name="repo-helper",
model="anthropic:claude-sonnet-4-6",
instructions="You are a careful repo assistant. Use your tools to answer from the code.",
tools=[*ai.toolkits.files(root="."), *ai.toolkits.git(root=".")],
)
result = Runner.run(agent, "What changed in the last commit? Summarize in 3 bullets.")
print(result.final_output)
This layer is where aisuite reaches beyond a pure API-routing library toward a production agent harness:
Tool policies—RequireApprovalPolicy
, allow/deny lists, or a custom callable deciding which tool calls are permitted to runState stores— persist and resume agent runs in memory, a file, or Postgres, continuing conversations across separate process runs** Artifacts & tracing**— capture what an agent produced and the steps it took to get there
MCP tools, natively #
client = ai.Client()
response = client.chat.completions.create(
model="openai:gpt-4o",
messages=[{"role": "user", "content": "List the files in the current directory"}],
tools=[{
"type": "mcp",
"name": "filesystem",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/directory"]
}],
max_turns=3
)
print(response.choices[0].message.content)
Installed via pip install 'aisuite[mcp]'
, MCP servers’ tools can be handed to any model with no manual schema wiring. For reusable connections, security filters, and tool-name prefixing across multiple MCP servers, aisuite exposes an explicit MCPClient
.
Extending aisuite: Adding a Provider # #
New providers plug in through a lightweight adapter with a fixed naming convention for automatic discovery:
| Element | Convention |
|---|---|
| Module file | <provider>_provider.py |
| Class name | <Provider>Provider (capitalized) |
class OpenaiProvider(BaseProvider):
...
aisuite vs. LiteLLM vs. a Direct SDK # #
| Aspect | aisuite | LiteLLM | Direct provider SDK |
|---|---|---|---|
| Unified chat API | Yes | Yes | No — one SDK per provider |
| Structured Agents API (Runner, toolkits, policies) | Yes | No (routing-focused) | No |
| Native MCP tool support | Yes | Partial (proxy-dependent) | No |
| State stores for resumable agents | Yes (memory/file/Postgres) | No | No |
| Powers a shipped desktop app | Yes (OpenWorker) | No | N/A |
| License | MIT | MIT | Varies by provider |
| Maintainer profile | Andrew Ng’s team | BerriAI | Provider itself |
The practical distinction: if you only need “call whichever LLM with one interface,” aisuite and LiteLLM solve the same problem in roughly the same shape. If you need an actual agent — tools, multi-turn execution, approval policies, resumable state — aisuite has that as a first-class layer rather than something you’d bolt on yourself.
Use Cases # #
1. Provider-Agnostic Application Code #
Write your chat logic once against client.chat.completions.create(...)
and swap model="openai:gpt-4o"
for model="anthropic:claude-sonnet-4-6"
without touching call sites — useful for A/B testing models or avoiding vendor lock-in.
2. Tool-Using Agents With Governance #
The RequireApprovalPolicy
and allow/deny-list tool policies matter once an agent has access to real tools (git, shell, files) — this is the layer that lets you grant capability without granting unchecked autonomy.
3. Resumable, Long-Running Agent Tasks #
Postgres-backed state stores mean an agent run can persist across process restarts — relevant for anything that shouldn’t lose progress if the host process dies mid-task.
4. Bridging Existing MCP Servers Into Any Model #
If you already run MCP servers (filesystem, custom internal tools), the native tools=[{"type": "mcp", ...}]
support means you don’t need a separate adapter layer to expose them to whichever LLM you’re calling that day.
Related Repositories # #
| Repository | Purpose |
|---|---|
Model Context Protocol## Related Articles #
LiteLLM — Unified OpenAI-Compatible API for 100+ LLM Providers— the closest direct comparison for the Chat Completions layer aloneLLM Gateway: Portkey vs LiteLLM vs OpenRouter— for weighing aisuite against the broader gateway/proxy category
Conclusion # #
aisuite covers familiar ground with its Chat Completions layer — provider abstraction is a crowded category — but backs it with a genuinely structured Agents API: tool policies, resumable state stores, and native MCP support, rather than routing alone. Built by Andrew Ng’s team, MIT licensed, actively committed to as of late July 2026, and validated by powering a real shipped product (OpenWorker), it’s a reasonable default if you want the provider-swapping convenience of a LiteLLM-style library but expect to grow into actual tool-using agents rather than staying at single-turn chat completions.
Best for: Python developers who want one interface across LLM providers today, with a credible growth path into governed, resumable agents without switching libraries later.
GitHub: https://github.com/andrewyng/aisuite
Last updated: 2026-08-02