Litelm: LiteLLM Without the Bloat

A new Python library called litelm extracts the core routing and message-translation functionality of litellm into roughly 2,900 lines of code with just 2 dependencies, openai and httpx, dropping the proxy server, caching, cost tracking, and Router class found in litellm's 100k+ line codebase. litelm routes LLM calls to 19 providers via a "provider/model-name" syntax, mirrors litellm's function names and response types, and offers async variants such as acompletion and aembedding. The library ships with extras for Anthropic and Bedrock, and switching from litellm requires only changing imports.

litellm's routing + translation in ~2,900 lines and 2 dependencies openai , httpx . litellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC of proxy servers, caching layers, cost tracking, and dozens of features most users never touch. litelm extracts just the call path — model routing, message translation, streaming, tool use, embeddings — and nothing else. No Router class, no proxy, no caching. pip install litelm openai + httpx pip install litelm anthropic + anthropic SDK pip install litelm bedrock + boto3 pip install litelm all everything python import litelm Basic completion response = litelm.completion "openai/gpt-4o", messages= {"role": "user", "content": "Hello "} print response.choices 0 .message.content Streaming for chunk in litelm.completion "groq/llama-3.1-70b-versatile", messages= ... , stream=True : print chunk.choices 0 .delta.content or "", end="" Embeddings response = litelm.embedding "openai/text-embedding-3-small", input= "hello world" Every function has an async variant: acompletion , aembedding , aresponses , atext completion . The API mirrors litellm — same function names, same arguments, same response types. If you're using litellm today, switching is s/litellm/litelm/ in your imports. | | litellm | litelm | |---|---|---| | Model routing provider/model → right endpoint | ✓ | ✓ | | Message translation Anthropic, Bedrock, Cloudflare, Mistral | ✓ | ✓ | | Streaming + stream chunk builder | ✓ | ✓ | | Tool use function calling | ✓ | ✓ | | Embeddings | ✓ | ✓ | | Text completions | ✓ | ✓ | | OpenAI Responses API | ✓ | ✓ | | Mock responses | ✓ | ✓ | | Router load balancing, fallbacks | ✓ | ✗ | | Proxy server | ✓ | ✗ | | Caching / budgeting / cost tracking | ✓ | ✗ | | Token counting | ✓ | ✗ | | Image gen, audio, OCR, fine-tuning | ✓ | ✗ | | Agents, guardrails, scheduler | ✓ | ✗ | Routes to 19 providers via "provider/model-name" syntax. Any OpenAI-compatible endpoint works via api base . | Provider | Env Var | Handler | Verified | |---|---|---|---| | OpenAI | OPENAI API KEY | OpenAI SDK | Yes | | Anthropic | ANTHROPIC API KEY | Custom | Yes | | Groq | GROQ API KEY | OpenAI-compat | Yes | | Mistral | MISTRAL API KEY | Custom | Yes | | xAI | XAI API KEY | OpenAI-compat | Yes | | OpenRouter | OPENROUTER API KEY | OpenAI-compat | Yes | | Azure | AZURE API KEY | OpenAI SDK Azure | Yes | | Bedrock | AWS ACCESS KEY ID | Custom | No | | Cloudflare | CLOUDFLARE API TOKEN | Custom | No | | Together | TOGETHERAI API KEY | OpenAI-compat | No | | Fireworks | FIREWORKS API KEY | OpenAI-compat | No | | DeepSeek | DEEPSEEK API KEY | OpenAI-compat | No | | Perplexity | PERPLEXITYAI API KEY | OpenAI-compat | No | | DeepInfra | DEEPINFRA API TOKEN | OpenAI-compat | No | | Gemini | GEMINI API KEY | OpenAI-compat | No | | Cohere | COHERE API KEY | OpenAI-compat | No | | Ollama | — | OpenAI-compat | No | | vLLM | — | OpenAI-compat | No | | LM Studio | — | OpenAI-compat | No | Set the environment variable for your provider: export OPENAI API KEY=sk-... export ANTHROPIC API KEY=sk-ant-... Or pass directly: litelm.completion "openai/gpt-4o", messages= ... , api key="sk-..." litelm.completion "openai/gpt-4o", messages= ... , api base="http://localhost:8000/v1" All provider errors are mapped to litelm's exception hierarchy: python from litelm import ContextWindowExceededError, RateLimitError, AuthenticationError try: response = litelm.completion "openai/gpt-4o", messages=messages except ContextWindowExceededError: prompt too long — truncate and retry pass except RateLimitError: back off pass except AuthenticationError: bad API key pass tools = {"type": "function", "function": { "name": "get weather", "parameters": {"type": "object", "properties": {"city": {"type": "string"}}}, }} response = litelm.completion "openai/gpt-4o", messages= {"role": "user", "content": "Weather in Paris?"} , tools=tools, tool choice="required", tool call = response.choices 0 .message.tool calls 0 print tool call.function.name, tool call.function.arguments Any OpenAI-compatible server works via api base : vLLM litelm.completion "openai/my-model", messages= ... , api base="http://localhost:8000/v1" Ollama litelm.completion "ollama/llama3", messages= ... , api base="http://localhost:11434/v1" LM Studio litelm.completion "openai/local-model", messages= ... , api base="http://localhost:1234/v1" litelm is human-directed, AI-assisted software. Much of the code was written with Claude Code using Claude Opus 4.6/4.7. Code written from 2026-05-14 onward is written through Pi using GPT-5.5. Compatibility claims are based on tests and maintainer review, not AI authorship. Maintainer attestation, 2026-09-11: LiteLLM's routing/formatting changes were reviewed from 649eb2d through 9a715df2 . The audit triaged 360 core-path commits, inspected upstream tests for potentially relevant behavior, and fixed the resulting compatibility gaps test-first. Local scoped tests: 262 passed, 55 skipped ; all 45 available-provider live tests and all 10 DSPy smoke tests also passed with the current dependency lock. This attests litelm's declared routing/formatting/DSPy surface only, not full litellm compatibility. Alpha. 262 own tests passing. The current scoped LiteLLM 9a715df2 baseline has 75 passing ported tests and no remaining actionable assertion/runtime failures. DSPy https://github.com/stanfordnlp/dspy drop-in verified — all 7 execution paths proven live Predict, CoT, typed signatures, streaming, embeddings, tool use, multi-output . uv run --extra all pytest tests/ -x --ignore=tests/ported --timeout=10 262 non-live tests bash scripts/ported contract.sh 49 fast upstream contract tests uv run --extra all pytest tests/test live.py -m live --timeout=30 45 live provider tests uv run pytest tests/test dspy smoke.py -m live --timeout=60 10 DSPy integration tests Live tests require API keys in .env.test . Skipped by default; run with -m live .