{"slug": "freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers", "title": "FreeLLMAPI: One OpenAI-Compatible Endpoint for 34 Free LLM Providers", "summary": "Developer Tashfeen Ahmed released FreeLLMAPI, an open-source router that unifies 34 free LLM providers behind a single OpenAI-compatible endpoint. The tool encrypts provider keys locally, tracks rate limits, and offers failover, with a paid tier for same-day model updates while the core software remains free.", "body_md": "Almost every AI lab now hands out a free tier. Google, Groq, Cerebras, Mistral, Cohere, NVIDIA, Cloudflare, OpenRouter, and a couple dozen more. Each one on its own is small. A few million tokens a month, a few thousand requests a day. Stacked together, they turn into something you can actually prototype against.\n\nThe problem is stacking them by hand. Thirty-four SDKs, thirty-four sets of rate limits, thirty-four places your request can fail at 2 AM.\n\n[FreeLLMAPI](https://github.com/tashfeenahmed/freellmapi) is an open-source router that collapses all of that into a single `/v1`\n\nendpoint. You point any OpenAI client at your own machine, and it routes across whatever providers you've added keys for.\n\nShort answer: yes, the software is free and stays free. There is a paid tier, but it does not gate the router.\n\nHere is the honest breakdown.\n\n**What is free, permanently**\n\n**What costs money**\n\nSo the mental model is: **the software is free, the tokens are free (they're your own free tiers), and the $19/yr is a convenience subscription for same-day model catalog updates.** If you're fine being a month behind on newly-launched free models, you never pay anything.\n\n**The catch that isn't about money**\n\nThe repo is blunt about this, and so am I: this is for personal experimentation and learning. Not production. No frontier models, variable latency, no SLA, and the effective quality of the endpoint drops late in the day as the best free models hit their daily caps (they reset at UTC midnight). Your relationship with each upstream provider is still governed by the terms you agreed to when you signed up for them. Ship something real, swap in a paid API first.\n\nYes, but probably not the way you're imagining. There is no signup page where FreeLLMAPI hands you a key to a hosted service. It is local-first and single-user by design.\n\nThe flow is two layers of keys:\n\n**Layer 1: your provider keys (inbound).** You go and get free-tier API keys yourself from Google AI Studio, Groq, Cerebras, Mistral, and so on. You paste them into the FreeLLMAPI dashboard on the **Keys** page. They get AES-256-GCM encrypted and stored in a local SQLite database, then decrypted in memory only for the duration of a request.\n\n**Layer 2: your unified key (outbound).** The router generates a single bearer token that looks like `freellmapi-...`\n\n. That is the only credential your applications ever see. Your app never touches the provider keys.\n\n```\nYour app  ──[freellmapi-xxx]──►  Local router  ──[real provider keys]──►  Groq\n                                      │                                   Google\n                                      │                                   Cerebras\n                                      └─ picks, tracks limits, fails over  ...\n```\n\n**What the router does per request:**\n\n`(provider, model, key)`\n\nso it stays under every free-tier cap instead of discovering the cap by getting rejected.`X-Routed-Via: <provider>/<model>`\n\nheader so you can see who actually served the request.There is also sticky sessions (a conversation stays on one model for 30 minutes so replies stay coherent), unified model entries when the same model exists on several providers, and named routing profiles you can switch per request with `auto:<profile>`\n\n.\n\nThe one-liner needs Docker. It creates `~/freellmapi`\n\n, generates an encryption key, pulls the image, and starts the container.\n\n```\ncurl -fsSL https://freellmapi.co/install.sh | bash\n```\n\nIf piping to bash makes you uncomfortable, the script is readable at the same URL first. Re-running it is safe: it keeps your existing `.env`\n\nand encryption key.\n\nOn Windows or macOS you can skip Docker entirely and grab the desktop installer from the GitHub Releases page. It runs the whole router and dashboard from your tray, and there is no password to set up.\n\n```\nhttp://localhost:3001\n```\n\nGo to the **Keys** page and paste in whatever free-tier keys you have. Start with two or three, you don't need all thirty-four. Good ones to begin with:\n\nEach key shows a status dot and when it was last health-checked, so you'll know immediately if you pasted a bad one.\n\nOn the **Models** page, drag your preferred models into order. That order is literally the routing priority. Top of the list gets tried first, and everything below it is your automatic failover.\n\nIt's in the header of the **Keys** page, starting with `freellmapi-`\n\n. This is the key your code uses.\n\nNothing new to learn. It's the OpenAI SDK with a different `base_url`\n\n.\n\n``` python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:3001/v1\",\n    api_key=\"freellmapi-your-unified-key\",\n)\n\nresp = client.chat.completions.create(\n    model=\"auto\",   # let the router choose\n    messages=[{\"role\": \"user\", \"content\": \"Explain database indexes in two sentences.\"}],\n)\n\nprint(resp.choices[0].message.content)\n```\n\nNode is the same idea:\n\n``` python\nimport OpenAI from \"openai\";\n\nconst client = new OpenAI({\n  baseURL: \"http://localhost:3001/v1\",\n  apiKey: \"freellmapi-your-unified-key\",\n});\n\nconst resp = await client.chat.completions.create({\n  model: \"auto\",\n  messages: [{ role: \"user\", content: \"Explain database indexes in two sentences.\" }],\n});\n\nconsole.log(resp.choices[0].message.content);\n```\n\nAnd plain curl:\n\n```\ncurl http://localhost:3001/v1/chat/completions \\\n  -H \"Authorization: Bearer freellmapi-your-unified-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"auto\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}'\n```\n\nFor the `model`\n\nfield you can pass:\n\n`\"auto\"`\n\n— router picks the best available`\"auto:fast\"`\n\nor `\"auto:smart\"`\n\n— bias toward speed or capability`\"auto:<your-profile>\"`\n\n— a named chain you built in the dashboard`\"fusion\"`\n\n— fan the prompt out to several free models in parallel and have a judge model synthesize one answerThis is where it gets genuinely useful. Most CLI agents configure themselves with one command:\n\n```\nnpx freellmapi setup-claude --url http://localhost:3001 --api-key <unified-key>\n```\n\nThere are generators for Codex CLI, Cline, Continue, Aider, OpenCode, Goose, Qwen Code, Roo, Kilo, Crush, DeepSeek Harness, and more. Every one supports `--dry-run`\n\nand backs up your existing config before touching it.\n\nBeyond `/v1/chat/completions`\n\n, the router implements:\n\n`/v1/responses`\n\n(what Codex CLI needs), `/v1/completions`\n\nfor editor ghost-text`/v1/embeddings`\n\n, `/v1/models`\n\n`/v1/images/generations`\n\n, `/v1/videos/generations`\n\n, `/v1/audio/speech`\n\n, `/v1/audio/transcriptions`\n\n`/v1/messages`\n\n— Anthropic's wire format, so Claude Code and the Anthropic SDKs work against your free pool`/v1beta`\n\n— Gemini's native surface for Gemini CLI`/mcp`\n\n— an MCP server so agents can introspect available models and provider health mid-sessionTool calling and structured outputs round-trip across providers, including a nice touch where plain-text tool calls from weaker models get rescued into proper `tool_calls`\n\n.\n\nYou can also add a **custom** provider pointing at any OpenAI-compatible endpoint: llama.cpp, LM Studio, vLLM, a local Ollama, or a remote gateway. So your local models sit in the same fallback chain as the cloud free tiers.\n\nUse it if you are: prototyping, building side projects, running a coding agent on your own machine, learning how routing and failover work, or just tired of managing eleven different `.env`\n\nvariables.\n\nDo not use it if you are: shipping to customers, need an SLA, need frontier-model quality, or need predictable latency.\n\nIt runs on anything with Node 20+, including a Raspberry Pi, at around 40 MB RSS idle.", "url": "https://wpnews.pro/news/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers", "canonical_source": "https://dev.to/arshtechpro/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers-3630", "published_at": "2026-09-04 09:01:45+00:00", "updated_at": "2026-09-04 09:24:13.557968+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "large-language-models"], "entities": ["FreeLLMAPI", "Tashfeen Ahmed", "Google", "Groq", "Cerebras", "Mistral", "Cohere", "NVIDIA"], "alternates": {"html": "https://wpnews.pro/news/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers", "markdown": "https://wpnews.pro/news/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers.md", "text": "https://wpnews.pro/news/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers.txt", "jsonld": "https://wpnews.pro/news/freellmapi-one-openai-compatible-endpoint-for-34-free-llm-providers.jsonld"}}