Comparing AI Agents Python Library Options for Production

A developer compares three Python libraries for building production AI agents: LangChain for flexibility, CrewAI for team-style orchestration, and LlamaIndex for data-centric retrieval. All three support OpenAI, Anthropic, and Azure endpoints with async APIs compatible with FastAPI, but differ in abstraction level and runtime cost.

If you need a Python library to build an AI agent that can run in production, start with LangChain for flexibility, CrewAI for team-style orchestration, or LlamaIndex if your focus is on data-centric retrieval. All three install with a single pip command, but they differ in abstraction level, runtime cost, and how easy they are to test inside a FastAPI service. The three most battle-tested options today are LangChain , CrewAI , and LlamaIndex . All three are open source, support OpenAI, Anthropic, and Azure endpoints, and have async APIs that play nicely with FastAPI. Installation is straightforward, but each library pulls a different set of optional dependencies. Core libraries pip install langchain crewai llama-index Optional LLM providers pick what you need pip install openai anthropic Vector store for LlamaIndex example pip install chromadb python from langchain.llms import OpenAI from langchain.prompts import PromptTemplate from langchain.chains import LLMChain llm = OpenAI model="gpt-4o-mini" prompt = PromptTemplate.from template "Translate this to French: {text}" chain = LLMChain llm=llm, prompt=prompt python from crewai import Agent, Task, Crew Define a simple research agent researcher = Agent role="Researcher", goal="Find the latest Python web frameworks", backstory="You are a senior backend engineer.", llm="gpt-4o-mini", task = Task description="List three frameworks with a one‑sentence summary each.", agent=researcher, crew = Crew agents= researcher , tasks= task python from llama index import VectorStoreIndex, SimpleDirectoryReader from llama index.llms import OpenAI documents = SimpleDirectoryReader "data/" .load data index = VectorStoreIndex.from documents documents, llm=OpenAI model="gpt-4o-mini" All three snippets run in a fresh virtual environment. The biggest gotcha I’ve hit is version mismatches between langchain and crewai when they both depend on openai . Pinning openai =1.0,<2.0 in requirements.txt saved me a day of debugging. Below is a tiny “weather lookup” agent that takes a city name, calls a mock weather API, and returns a friendly sentence. The logic is the same; only the orchestration differs. python import httpx from langchain.llms import OpenAI from langchain.prompts import PromptTemplate from langchain.chains import LLMChain llm = OpenAI model="gpt-4o-mini" prompt = PromptTemplate.from template "You are a helpful assistant. Use the JSON response {weather} to answer: " "What is the weather like in {city}?" async def fetch weather city: str - dict: async with httpx.AsyncClient as client: resp = await client.get f"https://api.example.com/weather?city={city}" return resp.json async def weather agent city: str - str: weather = await fetch weather city chain = LLMChain llm=llm, prompt=prompt return await chain.arun weather=weather, city=city python from crewai import Agent, Task, Crew import httpx weather agent = Agent role="Weather Bot", goal="Provide a short weather summary", backstory="You are a backend engineer who loves concise messages.", llm="gpt-4o-mini", async def fetch weather city : async with httpx.AsyncClient as client: r = await client.get f"https://api.example.com/weather?city={city}" return r.json task = Task description="Summarize the JSON weather data for {city}", expected output="A one‑sentence weather report.", agent=weather agent, async func=fetch weather, CrewAI lets you attach a callable crew = Crew agents= weather agent , tasks= task async def run crew city : return await crew.kickoff inputs={"city": city} python from llama index import SimpleDirectoryReader, VectorStoreIndex, ServiceContext from llama index.llms import OpenAI import httpx Build a trivial index containing a prompt template documents = SimpleDirectoryReader input files= "prompt.txt" .load data service context = ServiceContext.from defaults llm=OpenAI model="gpt-4o-mini" index = VectorStoreIndex.from documents documents, service context=service context async def fetch weather city : async with httpx.AsyncClient as client: r = await client.get f"https://api.example.com/weather?city={city}" return r.json async def llama agent city : query = f"Summarize this JSON: {await fetch weather city }" response = await index.as query engine .aquery query return response.response All three agents return a string like “In Paris it’s 12 °C with light rain.” The LangChain version feels most explicit; CrewAI hides the prompt plumbing but requires you to think in terms of tasks; LlamaIndex bundles the prompt inside an index, which can be overkill for a single API call. | Feature | LangChain | CrewAI | LlamaIndex | |---|---|---|---| Prompt control | Full, low-level | Medium templates per task | Low index-driven | Tool integration | Easy via Tool class | Built-in task-to-tool mapping | Requires custom retriever | Memory | Built-in ConversationBufferMemory | Implicit via task history | Index acts as persistent memory | Async support | Native async methods | Async tasks supported | Async query engine | Cost predictability | You manage each LLM call | Crew may fire extra calls for coordination | Index creation can add one-off token cost | Learning curve | Steeper, many concepts | Gentle, opinionated | Gentle if you only need retrieval | In my production services, LangChain adds ~15 ms per extra tool call, CrewAI adds ~30 ms overhead for task orchestration, and LlamaIndex adds a one-time indexing cost that can be several seconds for a 10 k document corpus. If you’re budget-conscious, watch out for CrewAI’s hidden “role-play” calls – they can double your token bill if you forget to set max tokens on the underlying LLM. FastAPI is already async-first, so you can drop any of the async agents directly into a route. Below is a minimal FastAPI app that uses the LangChain weather agent; swap the import for crew agent or llama agent to try the other libraries. python app/main.py from fastapi import FastAPI, HTTPException from pydantic import BaseModel import uvicorn Choose your agent implementation from agents.langchain weather import weather agent or crew agent, llama agent app = FastAPI title="Weather AI Agent" class CityRequest BaseModel : city: str @app.post "/weather" async def get weather req: CityRequest : try: result = await weather agent req.city return {"city": req.city, "report": result} except Exception as exc: raise HTTPException status code=502, detail=str exc if name == " main ": uvicorn.run "app.main:app", host="0.0.0.0", port=8000, reload=True httpx timeouts client = httpx.AsyncClient timeout=10.0 . OpenAI inside the route handler.If you need more context, see my earlier post on AI agent python ollama: Build, Test, Deploy with FastAPI https://www.logiclooptech.dev/ai-agent-python-ollama-build-test-deploy-with-fastapi/ for a full deployment pipeline. Testing LLM-driven code feels weird because the output is nondeterministic. Here’s what I rely on: assert "Translate this to French" in prompt . If the prompt changes, the test fails before you even call the model. OpenAI with a stub that returns a deterministic JSON payload. Libraries like pytest-mock let you patch OpenAI. call . This isolates the business logic from the provider. httpx.AsyncClient app=app in pytest to hit /weather with a fake city and assert the shape of the JSON response. max tokens parameter let the model keep streaming.When you’re ready to scale, read the “Testing and validation” chapter of the official LangChain docs – they outline a “LLMMock” helper that integrates nicely with pytest. Which library should I start with for a simple chatbot? LangChain gives the most flexibility and the smallest dependency footprint for a single-turn bot. If you want built-in task management, try CrewAI. Do I need a vector store for every agent? Only if you plan to retrieve or embed large text collections. LlamaIndex shines there; otherwise it adds unnecessary latency. Can I swap OpenAI for an on-prem model? All three libraries expose a generic LLM interface. Point the client to your local server e.g., OpenAI api base="http://localhost:8000/v1" and the rest of the code stays the same. How do I monitor token usage per request? Wrap the LLM call in a decorator that reads the usage field from the response and pushes it to Prometheus or CloudWatch. I keep a request id in the FastAPI state to correlate logs. pip install line; watch version pins for openai .