Free AI Models in Your CI Pipeline Will Fail Silently. Build the Circuit Breaker First.

A few months ago I wired a free AI coding model into a side project's CI pipeline. The job was modest: summarize each pull request diff into three bullet points for the changelog draft. It worked for eleven days. On day twelve, the model endpoint started returning empty completions with HTTP 200, and my pipeline happily committed twelve consecutive changelog entries that read, in full, "-". Nobody noticed for a week because the job was green.

That failure taught me something the demo-driven conversation around free AI models skips entirely: the problem with putting a zero-cost model into automation is not quality, it's silent degradation. A paid API with an SLA pages someone when it breaks. A free endpoint just gets weird, and your pipeline keeps shipping.

This article is the workflow I built after that incident. It's a small router with a circuit breaker and a canary quality gate, written in Python, that lets free model endpoints participate in CI automation without being able to fail silently. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I use MonkeyCode's free model access and its free server option as the concrete environment below, but the router is plain HTTP — it works against any OpenAI-compatible endpoint, which is the point.

Before writing code, it helps to enumerate how free endpoints actually fail in automation, because it's rarely a clean 500:

Notice what's missing: "the model gives a mediocre answer." For the class of tasks I'll argue free models belong in — changelogs, commit message linting, test-name generation, log summarization — mediocre-but-parseable is fine. Unparseable-and-committed is not.

The design has three moving parts:

ready

."). If the probe fails, the endpoint is degraded Here's a working minimal version. I've run this pattern against MonkeyCode's free server endpoint; the only configuration is the base URL and model name:

import time
import httpx

class ModelRouter:
    def __init__(self, base_url, api_key, model,
                 failure_threshold=3, cooldown_seconds=300):
        self.base_url = base_url.rstrip("/")
        self.api_key = api_key
        self.model = model
        self.failure_threshold = failure_threshold
        self.cooldown_seconds = cooldown_seconds
        self.consecutive_failures = 0
        self.circuit_opened_at = None

    def _circuit_open(self):
        if self.circuit_opened_at is None:
            return False
        if time.time() - self.circuit_opened_at > self.cooldown_seconds:
            return False
        return True

    def _record(self, ok):
        if ok:
            self.consecutive_failures = 0
            self.circuit_opened_at = None
        else:
            self.consecutive_failures += 1
            if self.consecutive_failures >= self.failure_threshold:
                self.circuit_opened_at = time.time()

    def _chat(self, messages, timeout=20):
        resp = httpx.post(
            f"{self.base_url}/v1/chat/completions",
            headers={"Authorization": f"Bearer {self.api_key}"},
            json={"model": self.model, "messages": messages,
                  "temperature": 0, "max_tokens": 512},
            timeout=timeout,
        )
        resp.raise_for_status()
        content = resp.json()["choices"][0]["message"]["content"]
        if not content or not content.strip():
            raise ValueError("empty completion with 200 status")
        return content

    def canary_ok(self):
        try:
            out = self._chat(
                [{"role": "user",
                  "content": "Reply with exactly the word: ready"}],
                timeout=10,
            )
            return out.strip().lower().rstrip(".") == "ready"
        except Exception:
            return False

    def complete(self, prompt):
        """Returns (text, source) where source is 'model' or 'fallback'."""
        if self._circuit_open():
            return None, "fallback"
        if not self.canary_ok():
            self._record(False)
            return None, "fallback"
        try:
            text = self._chat([{"role": "user", "content": prompt}])
            self._record(True)
            return text, "model"
        except Exception:
            self._record(False)
            return None, "fallback"

And the CI-side usage, which is where the safety property lives:

router = ModelRouter(
    base_url="https://your-monkeycode-server.example",  # free server endpoint
    api_key=os.environ["MC_API_KEY"],
    model=os.environ.get("MC_MODEL", "default"),
)

text, source = router.complete(f"Summarize this diff in 3 bullets:\n{diff}")

if source == "model":
    changelog_entry = text
else:
    changelog_entry = "- [auto-summary unavailable; see diff]"

write_changelog(changelog_entry)

The two details that matter most: the canary runs per invocation, not on a schedule, because free endpoints degrade minute-to-minute; and the empty-completion check in _chat

is what would have caught my original twelve-dash incident, since the endpoint never returned an error status.

This is the decision table I now apply before letting any free model touch automation:

The pattern: free models belong where the output is advisory or reviewed, the volume makes paid API costs annoying, and the fallback is cheap. The moment a wrong answer can reach production unreviewed, cost stops being the relevant variable.

The economics of free model access are genuinely useful — MonkeyCode's free models plus a free server option meant my changelog automation costs nothing to run, and the whole experiment was cheap to try. But "free to call" and "free to trust" are different statements, and CI is where the difference shows up at 2 a.m. as a green pipeline full of dashes. Build the breaker first, keep the fallback deterministic, and let the free tier earn its place in the pipeline the same way any flaky dependency does: behind a circuit that assumes it will fail.

If you want to try this pattern, the router above runs unmodified against MonkeyCode's free server — the canary prompt and fallback design are the parts worth copying, not the endpoint.