How I Cut My LLM Costs by 90% Without Changing My App Logic

How I Cut My LLM Costs by 90% Without Changing My App Logic There’s a particular kind of dread that comes with checking your OpenAI billing dashboard mid-month. I’ve been building a news automation hub that runs 14 editorial workspaces — summarizing, rewriting, fact-checking, SEO-tagging, and translation pipelines around the clock. The AI layer was already fairly optimized:

• Groq
• Gemini Flash
• DeepSeek
• OpenRouter
• provider rotation
• fallback logic But the final fallback was still OpenAI, and once rate limits hit, costs climbed faster than expected. What I needed wasn’t more routing logic. I needed a smarter endpoint. The Problem My setup already rotated between multiple providers, but the architecture had a weakness:

Provider exhausted
    -> fallback
        -> OpenAI
            -> credits disappear

The more providers I added, the messier things became:

• more API keys
• more retry logic
• more conditional branches
• more provider-specific handling I was optimizing infrastructure with application code. That was the mistake. The Fix After digging through self-hosted AI tooling, I found freellmapi . It’s a lightweight OpenAI-compatible proxy that automatically routes requests across multiple free-tier LLM providers:
• Groq
• Cerebras
• SambaNova
• Cloudflare Workers AI
• GitHub Models
• OpenRouter free models
• and others Combined free-tier capacity: roughly 800M tokens/month. The interesting part is that the routing happens inside the proxy — not inside your app. My Integration The integration took less than an hour.

1. Deploy the proxy I ran it on my existing VPS:

• Node.js 20
• ~40MB idle RAM
• localhost only

2. Add provider credentials I added:

• Groq key
• Cloudflare credentials
• OpenRouter key inside the admin panel.

3. Point my app to a single endpoint

const client = new OpenAI({
  baseURL: "http://localhost:3001/v1",
  apiKey: process.env.LOCAL_ROUTER_KEY
});

That was basically it. The important detail: I stopped specifying models for non-critical tasks. Instead of forcing a specific provider, I let the proxy auto-route requests to whatever free provider was currently available.

App
  -> freellmapi
      -> Groq
      -> Cloudflare Workers AI
      -> Cerebras
      -> SambaNova
      -> OpenRouter

If Groq rate-limited:

• another provider picked up the request If a provider became slow:
• routing shifted automatically My application code never needed to know. The Result Within 24 hours:
• OpenAI usage dropped by ~90%
• background AI tasks became almost entirely free-tier
• no additional retry logic was needed Most importantly: I removed provider chaos from my application layer. What I Learned When engineers hit rate limits, the instinct is usually:
• add more providers
• add more fallback logic
• add more code But sometimes the better solution is adding an abstraction layer that absorbs the complexity for you. Another realization: Most AI tasks do not require a specific premium model. For:
• summaries
• tagging
• drafts
• translations
• background enrichment …almost any decent modern 70B model works fine. Caveats Free-tier infrastructure has tradeoffs. Some providers:
• have cold starts
• introduce latency spikes
• become temporarily unavailable For real-time user-facing chat systems, you should test failover carefully. For async pipelines and batch jobs, though, it’s been surprisingly solid. Also: run this on infrastructure you control. A proxy like this handles upstream API keys — don’t hand that responsibility to random hosted services. Final Thought The biggest optimization wasn’t changing models. It was removing complexity from the layer that had to manage them.