Bypass AI API Rate Limits & Achieve 100% Uptime with SmartCore LLM Proxy

Hitting HTTP 429 Rate Limits on Google Gemini, DeepSeek, or OpenAI can crash your AI backend or assistant (like Hermes AI).

SmartCore LLM Proxy is a lightweight local gateway built with LiteLLM that automatically load-balances requests and routes to fallback models whenever your primary key hits a limit.

Instead of calling LLM APIs directly, your app points to http://127.0.0.1:4000

:

Instead of calling LLM APIs directly, your app points to http://127.0.0.1:4000

:

[ App / Hermes AI ] ──▶ [ SmartCore LLM Proxy ] │ ┌───────────────────────┴───────────────────────┐ ▼ ▼ Gemini 1.5 Pro ──(On 429)──▶ Gemini 1.5 Flash / DeepSeek

bash
git clone https://github.com/aaserag1/SmartCore-LLM-Proxy.git
cd SmartCore-LLM-Proxy
pip install "litellm[proxy]"
2. Configure Keys (config.yaml)
Add your API keys and fallbacks in config.yaml:

yaml

model_list:
  - model_name: gemini-1.5-pro
    litellm_params:
      model: gemini/gemini-1.5-pro
      api_key: "YOUR_GOOGLE_KEY"
  - model_name: deepseek-coder
    litellm_params:
      model: deepseek/deepseek-chat
      api_key: "YOUR_DEEPSEEK_KEY"
router_settings:
  cooldown_time: 60
  fallbacks: 
    - gemini-1.5-pro: ["deepseek-coder"]
3. Run Gateway
Double-click Run_LiteLLM.bat or run:

bash

python -m litellm --config config.yaml --port 4000 --host 127.0.0.1
4. Connect Your App
Set your app's Base URL to http://127.0.0.1:4000.

🔗 Open Source & Contribution
Check out the project and leave a ⭐ on GitHub!

🐙 GitHub: https://github.com/aaserag1/SmartCore-LLM-Proxy
👨‍💻 Author: Ahmed Adel (Abo Adel)
5:09 AM