{"slug": "litelm-litellm-without-the-bloat", "title": "Litelm: LiteLLM Without the Bloat", "summary": "A new Python library called litelm extracts the core routing and message-translation functionality of litellm into roughly 2,900 lines of code with just 2 dependencies, openai and httpx, dropping the proxy server, caching, cost tracking, and Router class found in litellm's 100k+ line codebase. litelm routes LLM calls to 19 providers via a \"provider/model-name\" syntax, mirrors litellm's function names and response types, and offers async variants such as acompletion and aembedding. The library ships with extras for Anthropic and Bedrock, and switching from litellm requires only changing imports.", "body_md": "litellm's routing + translation in ~2,900 lines and 2 dependencies (`openai`, `httpx`).\n\nlitellm routes LLM calls across providers and translates between message formats. That core is buried under 100k+ LOC of proxy servers, caching layers, cost tracking, and dozens of features most users never touch. litelm extracts just the call path — model routing, message translation, streaming, tool use, embeddings — and nothing else. No Router class, no proxy, no caching.\n\n```\npip install litelm                # openai + httpx\npip install litelm[anthropic]     # + anthropic SDK\npip install litelm[bedrock]       # + boto3\npip install litelm[all]           # everything\npython\nimport litelm\n\n# Basic completion\nresponse = litelm.completion(\"openai/gpt-4o\", messages=[{\"role\": \"user\", \"content\": \"Hello!\"}])\nprint(response.choices[0].message.content)\n\n# Streaming\nfor chunk in litelm.completion(\"groq/llama-3.1-70b-versatile\", messages=[...], stream=True):\n    print(chunk.choices[0].delta.content or \"\", end=\"\")\n\n# Embeddings\nresponse = litelm.embedding(\"openai/text-embedding-3-small\", input=[\"hello world\"])\n```\n\nEvery function has an async variant: `acompletion`, `aembedding`, `aresponses`, `atext_completion`.\n\nThe API mirrors litellm — same function names, same arguments, same response types. If you're using litellm today, switching is `s/litellm/litelm/` in your imports.\n\n|  | litellm | litelm | \n|---|---|---|\n| Model routing ( `provider/model` → right endpoint) | ✓ | ✓ | \n| Message translation (Anthropic, Bedrock, Cloudflare, Mistral) | ✓ | ✓ | \n| Streaming + `stream_chunk_builder` | ✓ | ✓ | \n| Tool use (function calling) | ✓ | ✓ | \n| Embeddings | ✓ | ✓ | \n| Text completions | ✓ | ✓ | \n| OpenAI Responses API | ✓ | ✓ | \n| Mock responses | ✓ | ✓ | \n| Router (load balancing, fallbacks) | ✓ | ✗ | \n| Proxy server | ✓ | ✗ | \n| Caching / budgeting / cost tracking | ✓ | ✗ | \n| Token counting | ✓ | ✗ | \n| Image gen, audio, OCR, fine-tuning | ✓ | ✗ | \n| Agents, guardrails, scheduler | ✓ | ✗ | \n\nRoutes to 19 providers via `\"provider/model-name\"` syntax. Any OpenAI-compatible endpoint works via `api_base`.\n\n| Provider | Env Var | Handler | Verified | \n|---|---|---|---|\n| OpenAI | `OPENAI_API_KEY` | OpenAI SDK | Yes | \n| Anthropic | `ANTHROPIC_API_KEY` | Custom | Yes | \n| Groq | `GROQ_API_KEY` | OpenAI-compat | Yes | \n| Mistral | `MISTRAL_API_KEY` | Custom | Yes | \n| xAI | `XAI_API_KEY` | OpenAI-compat | Yes | \n| OpenRouter | `OPENROUTER_API_KEY` | OpenAI-compat | Yes | \n| Azure | `AZURE_API_KEY` | OpenAI SDK (Azure) | Yes | \n| Bedrock | `AWS_ACCESS_KEY_ID` | Custom | No | \n| Cloudflare | `CLOUDFLARE_API_TOKEN` | Custom | No | \n| Together | `TOGETHERAI_API_KEY` | OpenAI-compat | No | \n| Fireworks | `FIREWORKS_API_KEY` | OpenAI-compat | No | \n| DeepSeek | `DEEPSEEK_API_KEY` | OpenAI-compat | No | \n| Perplexity | `PERPLEXITYAI_API_KEY` | OpenAI-compat | No | \n| DeepInfra | `DEEPINFRA_API_TOKEN` | OpenAI-compat | No | \n| Gemini | `GEMINI_API_KEY` | OpenAI-compat | No | \n| Cohere | `COHERE_API_KEY` | OpenAI-compat | No | \n| Ollama | — | OpenAI-compat | No | \n| vLLM | — | OpenAI-compat | No | \n| LM Studio | — | OpenAI-compat | No | \n\nSet the environment variable for your provider:\n\n```\nexport OPENAI_API_KEY=sk-...\nexport ANTHROPIC_API_KEY=sk-ant-...\n```\n\nOr pass directly:\n\n```\nlitelm.completion(\"openai/gpt-4o\", messages=[...], api_key=\"sk-...\")\nlitelm.completion(\"openai/gpt-4o\", messages=[...], api_base=\"http://localhost:8000/v1\")\n```\n\nAll provider errors are mapped to litelm's exception hierarchy:\n\n``` python\nfrom litelm import ContextWindowExceededError, RateLimitError, AuthenticationError\n\ntry:\n    response = litelm.completion(\"openai/gpt-4o\", messages=messages)\nexcept ContextWindowExceededError:\n    # prompt too long — truncate and retry\n    pass\nexcept RateLimitError:\n    # back off\n    pass\nexcept AuthenticationError:\n    # bad API key\n    pass\ntools = [{\"type\": \"function\", \"function\": {\n    \"name\": \"get_weather\",\n    \"parameters\": {\"type\": \"object\", \"properties\": {\"city\": {\"type\": \"string\"}}},\n}}]\n\nresponse = litelm.completion(\n    \"openai/gpt-4o\", messages=[{\"role\": \"user\", \"content\": \"Weather in Paris?\"}],\n    tools=tools, tool_choice=\"required\",\n)\ntool_call = response.choices[0].message.tool_calls[0]\nprint(tool_call.function.name, tool_call.function.arguments)\n```\n\nAny OpenAI-compatible server works via `api_base`:\n\n```\n# vLLM\nlitelm.completion(\"openai/my-model\", messages=[...], api_base=\"http://localhost:8000/v1\")\n\n# Ollama\nlitelm.completion(\"ollama/llama3\", messages=[...], api_base=\"http://localhost:11434/v1\")\n\n# LM Studio\nlitelm.completion(\"openai/local-model\", messages=[...], api_base=\"http://localhost:1234/v1\")\n```\n\nlitelm is human-directed, AI-assisted software. Much of the code was written with Claude Code using Claude Opus 4.6/4.7. Code written from 2026-05-14 onward is written through Pi using GPT-5.5. Compatibility claims are based on tests and maintainer review, not AI authorship.\n\nMaintainer attestation, 2026-09-11: LiteLLM's routing/formatting changes were reviewed from `649eb2d` through `9a715df2`. The audit triaged 360 core-path commits, inspected upstream tests for potentially relevant behavior, and fixed the resulting compatibility gaps test-first. Local scoped tests: `262 passed, 55 skipped`; all 45 available-provider live tests and all 10 DSPy smoke tests also passed with the current dependency lock.\n\nThis attests litelm's declared routing/formatting/DSPy surface only, not full litellm compatibility.\n\n**Alpha.** 262 own tests passing. The current scoped LiteLLM `9a715df2` baseline has 75 passing ported tests and no remaining actionable assertion/runtime failures.\n\n[DSPy](https://github.com/stanfordnlp/dspy) drop-in verified — all 7 execution paths proven live (Predict, CoT, typed signatures, streaming, embeddings, tool use, multi-output).\n\n```\nuv run --extra all pytest tests/ -x --ignore=tests/ported --timeout=10  # 262 non-live tests\nbash scripts/ported_contract.sh                                        # 49 fast upstream contract tests\nuv run --extra all pytest tests/test_live.py -m live --timeout=30       # 45 live provider tests\nuv run pytest tests/test_dspy_smoke.py -m live --timeout=60             # 10 DSPy integration tests\n```\n\nLive tests require API keys in `.env.test`. Skipped by default; run with `-m live`.", "url": "https://wpnews.pro/news/litelm-litellm-without-the-bloat", "canonical_source": "https://github.com/kennethwolters/litelm", "published_at": "2026-09-11 18:10:20+00:00", "updated_at": "2026-09-11 18:44:26.537084+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-infrastructure"], "entities": ["litelm", "litellm", "OpenAI", "Anthropic", "Bedrock", "Groq", "httpx", "boto3"], "alternates": {"html": "https://wpnews.pro/news/litelm-litellm-without-the-bloat", "markdown": "https://wpnews.pro/news/litelm-litellm-without-the-bloat.md", "text": "https://wpnews.pro/news/litelm-litellm-without-the-bloat.txt", "jsonld": "https://wpnews.pro/news/litelm-litellm-without-the-bloat.jsonld"}}