Bypass AI API Rate Limits & Achieve 100% Uptime with SmartCore LLM Proxy

Developer Ahmed Adel (Abo Adel) released SmartCore LLM Proxy, an open-source local gateway built with LiteLLM that automatically load-balances requests and routes to fallback models when primary API keys hit rate limits from providers like Google Gemini, DeepSeek, or OpenAI. The tool aims to prevent HTTP 429 errors from crashing AI backends or assistants.

Hitting HTTP 429 Rate Limits on Google Gemini, DeepSeek, or OpenAI can crash your AI backend or assistant like Hermes AI . SmartCore LLM Proxy is a lightweight local gateway built with LiteLLM that automatically load-balances requests and routes to fallback models whenever your primary key hits a limit. Instead of calling LLM APIs directly, your app points to http://127.0.0.1:4000 : Instead of calling LLM APIs directly, your app points to http://127.0.0.1:4000 : App / Hermes AI ──▶ SmartCore LLM Proxy │ ┌───────────────────────┴───────────────────────┐ ▼ ▼ Gemini 1.5 Pro https://dev.toPrimary ── On 429 ──▶ Gemini 1.5 Flash / DeepSeek https://dev.toFallback bash git clone https://github.com/aaserag1/SmartCore-LLM-Proxy.git cd SmartCore-LLM-Proxy pip install "litellm proxy " 2. Configure Keys config.yaml Add your API keys and fallbacks in config.yaml: yaml model list: - model name: gemini-1.5-pro litellm params: model: gemini/gemini-1.5-pro api key: "YOUR GOOGLE KEY" - model name: deepseek-coder litellm params: model: deepseek/deepseek-chat api key: "YOUR DEEPSEEK KEY" router settings: cooldown time: 60 fallbacks: - gemini-1.5-pro: "deepseek-coder" 3. Run Gateway Double-click Run LiteLLM.bat or run: bash python -m litellm --config config.yaml --port 4000 --host 127.0.0.1 4. Connect Your App Set your app's Base URL to http://127.0.0.1:4000. 🔗 Open Source & Contribution Check out the project and leave a ⭐ on GitHub 🐙 GitHub: https://github.com/aaserag1/SmartCore-LLM-Proxy 👨‍💻 Author: Ahmed Adel Abo Adel 5:09 AM