# Learn Provider-Agnostic Model Routing by Building a Tiny LLM Switchboard > Source: > Published: 2026-08-13 03:04:50+00:00 Every few weeks a new model drops and my study group chat fills up with screenshots: "this one is cheaper," "this one is better at code," "switch now." I can never verify any of it quickly, because my test scripts all hard-code one provider's client. Rewriting the harness is slower than the hype cycle. So here is the learning question: **can a ~60-line, standard-library-only Python switchboard let me swap providers behind one interface, route toy tasks to different backends, and prove with a failing fixture where the routing breaks?** Run the final script and you should see: ``` [route=summarize] backend=local-echo cost=0.0000 out='SUMMARY: ...' [route=code] backend=mock-strong cost=0.0020 out='def add(a, b): ...' [route=summarize] backend=mock-strong cost=0.0020 out='SUMMARY: ...' (fallback fired) ``` The third line is the interesting one — keep reading. A wrapper hides one provider's quirks. A **switchboard** does something different: it defines *your* task types (`summarize` , `code` , `chat` ) and maps each to a backend plus a fallback. When a new model appears — whatever it is called this month — you add one entry and re-run the same fixtures. Your evaluation questions stay fixed while backends rotate under them. That is the actual skill: separating *what you ask* from *who answers*. Save as `switchboard.py` : ``` """Tiny provider-agnostic LLM switchboard (stdlib only).""" from dataclasses import dataclass, field from typing import Callable # --- Backends: same signature (str) -> str, so they are interchangeable --- def local_echo(prompt: str) -> str: """Free 'backend': deterministic, offline, good enough for smoke tests.""" return f"SUMMARY: {prompt[:40]}" def mock_strong(prompt: str) -> str: """Pretend paid model. Fails on empty input, like a real API rejects it.""" if not prompt.strip(): raise ValueError("backend rejected empty prompt") if prompt.startswith("write code"): return "def add(a, b):\n return a + b" return f"SUMMARY: {prompt[:40]}" @dataclass class Backend: name: str fn: Callable[[str], str] cost_per_call: float # USD, your own pricing notes go here @dataclass class Switchboard: routes: dict[str, list[Backend]] = field(default_factory=dict) spent: float = 0.0 def register(self, task: str, backends: list[Backend]) -> None: self.routes[task] = backends def run(self, task: str, prompt: str) -> str: if task not in self.routes: raise KeyError(f"no route registered for task '{task}'") last_err = None for backend in self.routes[task]: try: out = backend.fn(prompt) self.spent += backend.cost_per_call print(f"[route={task}] backend={backend.name:<11} " f"cost={backend.cost_per_call:.4f} out={out.splitlines()[0]!r}") return out except Exception as e: # try the fallback backend last_err = e raise RuntimeError(f"all backends failed for '{task}'") from last_err if __name__ == "__main__": free = Backend("local-echo", local_echo, 0.0) paid = Backend("mock-strong", mock_strong, 0.002) sb = Switchboard() sb.register("summarize", [free, paid]) # cheap first, paid as fallback sb.register("code", [paid]) # only the 'strong' backend sb.run("summarize", "explain gradient descent to a first-year student") sb.run("code", "write code to add two numbers") sb.run("summarize", " ") # free backend accepts junk... or does it? print(f"total spent: ${sb.spent:.4f}") [route=summarize] backend=local-echo cost=0.0000 out='SUMMARY: explain gradient descent to a f' [route=code] backend=mock-strong cost=0.0020 out='def add(a, b):' [route=summarize] backend=local-echo cost=0.0000 out='SUMMARY: ' total spent: $0.0020 ``` Wait — the third line did **not** fall back, and it returned a garbage summary of whitespace. My `local_echo` backend happily accepts an empty prompt. The failure I promised in the intro only fires if the *primary* backend raises. Before reading on: which fixture input would force the fallback line from the intro to appear? (Answer at the bottom.) Swap the registration so the strict backend is primary: ``` sb.register("summarize", [paid, free]) # strict first, lenient as fallback sb.run("summarize", " ") ``` Now you get the intro's third line: `mock-strong` raises on the empty prompt, the switchboard catches it, and `local-echo` answers instead. The concept that actually matters: **fallback order is a policy decision, and "free first" and "strict first" fail in opposite directions.** Free-first silently returns junk; strict-first silently spends money when you expected the cheap path. `Exception` is fine for a teaching harness, but in real code you should distinguish "provider is down" (retry/fallback) from "my prompt was rejected" (fallback just re-fails expensively).I do most of these experiments inside MonkeyCode, since its free model access lets me prototype against a real LLM endpoint instead of only mocks, and the free server option means the harness runs somewhere that is not my laptop between classes. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The switchboard pattern above is deliberately provider-agnostic, though — the same file works against any endpoint, including none at all. Add a third route, `"chat"` , and a fixture file of five prompts where you *predict in writing* which backend should handle each one. Then make `mock_strong` randomly raise on 30% of calls (seed `random` for reproducibility) and check whether your predictions about cost still hold. If your cost estimate assumed zero failures, what does that tell you about launch-week pricing comparisons? With `summarize` registered as `[free, paid]` , only an input that makes `local_echo` itself raise would trigger the fallback — and `local_echo` never raises. The intro's third line only appears under the strict-first registration. If you predicted that, you understood the policy-ordering point; if not, run both versions and diff the output. If you find a fixture input where the fallback makes things *worse* (e.g., the lenient backend returns something dangerously plausible), post it — minimal counterexamples are the best part of these threads.