Our team used multiple LLM providers from different clouds: Google, Azure, and AWS. Every app and project had its own API key. We could not see which key belonged to which app, team, env etc. We also had no clear view of token usage and tracking. Later, finance asked which product spent a lot of mon
Our team used multiple LLM providers from different clouds: Google, Azure, and AWS. Every app and project had its own API key. We could not see which key belonged to which app, team, env etc. We also had no clear view of token usage and tracking. Later, finance asked which product spent a lot of money. We could not answer. First I tried LiteLLM. It is a good tool if you need to quickly setup your llm gateway. Then the team became larger. LiteLLM Admin UI SSO is free for only 5 users. After 5 users, you need the Enterprise plan. Org admin and SCIM are also Enterprise. I did not want a license wall on the control plane. So I built Open LLM Gateway and released the full stack as Apache 2.0. That includes the admin UI, not only the proxy. What is in the repo: OpenAI /openai/* and Anthropic /anthropic/* proxy Encrypted master keys; apps use child keys (sk_…) Portal: organizations, roles, invites, optional Google login, key rotate / tag / disable Cost and token analytics by provider, model, and tag Deploy on Cloudflare Workers and Vercel There is no 5-user SSO seat cap in the open-source project. What it is not: it is not LiteLLM if you only need limited seat without enterprise features. It is not OpenRouter if you want a hosted service and you do not want to run it yourself. Spend limits and guardrails are not ready yet. If your company wants a self-hosted LLM gateway, with no seat cap, and with the admin UI fully open source, Open LLM Gateway may be a good fit. Source: https://github.com/Nelsonlin0321/llm-gateway Site: https://llm-gateway.io
Key Takeaways #
- •Our team used multiple LLM providers from different clouds: Google, Azure, and AWS
- •This story was reported by Dev.to, covering developments in the** dev**space. - •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.
📖 Continue reading the full article: