I Moved My Discord Bot Off a Paid LLM API. Five Things Broke.

A developer who runs a Discord bot that rewrites stack traces into plain English moved it from a paid LLM API to a free endpoint and documented five failures that emerged over two weeks. The migration exposed issues with hardcoded model names, rate-limit handling, token counting, and response parsing, leading the developer to build a wrapper for provider portability. The developer verified the issues were generic by reproducing two against a second endpoint.

My Discord bot has one job: when someone pastes a stack trace into our server's help channel, it rewrites the trace into a plain-English explanation plus a likely fix. It ran for months on a paid LLM API at a few dollars a month — small money, but the kind that nags you when the workload is trivially bursty: silent for days, then thirty requests in an evening when a game update breaks everyone's mods. So I pointed it at a free endpoint instead. The migration took an afternoon. The consequences of the migration took two weeks to fully shake out, because nothing failed loudly. Everything failed slightly. This is a log of the five things that broke, what each one taught me about provider portability, and the wrapper code that now sits between my bot and any LLM endpoint so I never have to debug these twice. The endpoint I moved to is MonkeyCode https://monkeycode.dev , which offers free model access and a free server option — the right price shape for a hobby bot with spiky traffic. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Every failure below, though, is generic to switching OpenAI-compatible providers. I verified that by deliberately reproducing two of them against a second endpoint. Treat this as a portability field report, not a product review. My config had the model name hardcoded in three places: the request builder, the logging line, and a cost-estimation function that divided tokens by a per-model price. When I swapped providers, I updated one of the three. The bot ran fine — and my logs confidently recorded the old model name for a week, which made every later debugging session confusing because I was reading telemetry for a model I wasn't using. Fix: model identity is configuration, not code, and it flows from exactly one source. python llm config.py — single source of truth from dataclasses import dataclass import os @dataclass frozen=True class LLMConfig: base url: str api key: str model: str whatever the CURRENT provider's catalog calls it max output tokens: int timeout s: float def load config - LLMConfig: return LLMConfig base url=os.environ "LLM BASE URL" , api key=os.environ "LLM API KEY" , model=os.environ "LLM MODEL" , max output tokens=int os.environ.get "LLM MAX OUTPUT", "600" , timeout s=float os.environ.get "LLM TIMEOUT S", "20" , Boring, obvious, and the root cause of the messiest week. A related trap: never assume a model name means the same thing across providers, or even across months on the same provider. Catalogs change. Check what your endpoint actually serves before writing it into an environment variable, and expect to change it again. On the paid API, I had never once seen a 429. On a free tier, the evening burst pattern hit the limit within the first weekend. The truly bad part: my error handler treated non-200 responses as "log and move on," so during peak hours the bot silently ignored half the stack traces posted. Users thought it was ignoring them specifically and reposted, which made the burst worse. A rate limit had become a feedback loop. Fix: bounded retries with jittered backoff, and — critically — a visible degraded state. python llm call.py — retry wrapper with honest degradation import asyncio import random import openai MAX ATTEMPTS = 4 async def call with backoff client, cfg, messages - str | None: for attempt in range MAX ATTEMPTS : try: resp = await asyncio.to thread client.chat.completions.create, model=cfg.model, messages=messages, temperature=0.2, max tokens=cfg.max output tokens, timeout=cfg.timeout s, return resp.choices 0 .message.content except openai.RateLimitError: if attempt == MAX ATTEMPTS - 1: return None caller must say so out loud base = min 2 attempt, 8 await asyncio.sleep base + random.uniform 0, base except openai.APITimeoutError: return None a slow answer in a chat channel is a wrong answer return None async def explain trace client, cfg, trace: str - str: result = await call with backoff client, cfg, {"role": "system", "content": SYSTEM PROMPT}, {"role": "user", "content": trace}, if result is None: return "⚠️ I'm rate-limited right now. Try again in a minute, " "or trim the trace to the first error line — shorter " "messages are more likely to get through." return result The degraded message does two jobs: it resets user expectations, and it gives them an actionable workaround shorter input . Silence is the worst possible response to a rate limit in an interactive product. The old setup had headroom I never measured, so my prompt builder happily concatenated the system prompt, the full stack trace, and the last five channel messages for context. On the new model, long traces started returning explanations of the wrong error — because the actual exception line, at the end of the paste, had been truncated away by the context window. No exception was raised. The model just answered a different question than the user asked. This is the nastiest class of provider-migration bug: output stays plausible while becoming wrong. Fix: budget tokens explicitly and truncate deliberately, keeping the semantically important part. For stack traces, the important part is the last exception block, not the first frames: php def trim trace trace: str, char budget: int - str: """Keep the tail of the trace: the exception line matters most.""" if len trace <= char budget: return trace head room = char budget // 4 tail = trace - char budget - head room : don't cut mid-line; align to a newline nl = tail.find "\n" if 0 < nl < 200: tail = tail nl + 1: return trace :head room + "\n ...middle frames omitted... \n" + tail Character budgets are a crude proxy for tokens, but a 4:1 char-to-token estimate with a safety margin beats not thinking about it at all. The general principle: after any provider switch, find your longest realistic input and verify end-to-end what the model actually receives. The bot edits its own Discord message progressively as tokens arrive — a nice touch that made it feel fast. The new endpoint delivered chunks on a different cadence: larger, less frequent batches. The visual result was a message that sat empty for seconds, then jumped. Worse, my progressive-edit logic issued a Discord API edit per chunk, and the chunkier cadence interacted badly with Discord's own rate limits on message edits. Fix: I removed streaming for this use case. Deliberate regression, correct call. For a chat bot whose answers are a few hundred tokens, perceived speed comes more from a fast first reaction than from progressive rendering. The bot now posts "🔎 reading the trace…" immediately a fast, guaranteed reaction , then replaces it with the full answer in one edit. Users rated it as feeling faster than the stuttering stream. The lesson generalizes: provider features that seem free — streaming, JSON modes, tool calling — are exactly where behavioral differences hide, because they're the least standardized parts of an API that is otherwise deliberately compatible. I had an uptime monitor hitting the endpoint's root URL. It stayed green through every incident above, because the server was always up ; it was the model path that rate-limited, truncated, or lagged. A TCP-level health check for an LLM-backed feature is close to useless. Fix: a synthetic probe that exercises the real path, run on a schedule, alerting on latency and content sanity: php async def synthetic probe client, cfg - dict: start = asyncio.get event loop .time result = await call with backoff client, cfg, {"role": "user", "content": "Reply with exactly: PROBE OK"}, elapsed = asyncio.get event loop .time - start return { "ok": result is not None and "PROBE OK" in result, "latency s": round elapsed, 2 , } Two weeks of probe data also gave me something I never had on the paid API: an honest picture of the free tier's behavior under my real traffic pattern — when bursts collide with limits, what p50 latency actually feels like. If you run one thing from this article, run the probe. It converts "the free tier seems flaky" into data you can act on. The switch was worth it: the bot's operating cost for this workload went to zero, and every failure above was a latent bug in my code that the paid tier's headroom had been papering over. Free capacity is unforgiving in a useful way. But be honest about the fit: If you want a concrete place to try this, MonkeyCode's free model access and free server option are what this bot currently runs on; the wrapper above means it could run somewhere else tomorrow, which is precisely the property worth engineering for. The migration took an afternoon. The engineering to make the next migration boring took two weeks — and it's the part that was actually worth doing.