Free LLM Tiers Are Lying to You — Model Rotation Saved My Agent

I built a small autonomous agent (GitHub issue triage) that runs entirely on free LLM tiers — no API budget, no GPU, no AWS account. The project itself is ~250 lines and works. The interesting part is what the free tier did to me.

The plan was simple: every open issue → LLM → JSON verdict. First real run on a big repo: almost every call failed. Not because the prompt was bad — because the model endpoint answered with 503 Service Unavailable

. Free endpoints on popular models are chronically overloaded; they accept the request and then just… don't.

This isn't hypothetical. My scan of facebook/react

(22 items) hit 503s repeatedly; the "free" model that the docs told me to use was effectively down half the time.

The agent now holds an ordered list of models and tries them one by one until one answers:

const MODELS = [
  'liquid/lfm-2.5-2.6b:free',          // fast, usually up
  'thinkingmachines/inkling-small:free', // backup
  'nvidia/nemotron-3.5-lightning:free',
  'cohere/north-mini-code:free',
]
for (const model of MODELS) {
  try { return await call(model, prompt) } catch {}
}

Semantics matter: any non-2xx (or empty) response is treated as a failure and we move to the next model. No retries-with-delay loops — just instant failover. Results before/after:

facebook/react

scan → large fraction of llm 503

errors, dashboard full of red.{"category","short","priority"}

and my parser takes the first {...}

block, tolerating markdown noise. Small models return sloppy JSON — the contract still wins.Free models are slower and sometimes dumber than paid ones. For {category, short, priority}

— a constrained classification task — the difference is irrelevant. For open-ended reasoning, it isn't. Match the model class to the task.

If you're building agents on free tiers: your first engineering task isn't the prompt — it's failover. Rotate, parse defensively, and design for 503s. Then the free tier becomes genuinely free, and your demo stops breaking mid-scan.

I put the whole thing (agent loop + rotation + no-framework dashboard) in a public repo: github.com/pyfile-toolkit/agent-triage. Have fun.