{"slug": "i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke", "title": "I Moved My Discord Bot Off a Paid LLM API. Five Things Broke.", "summary": "A developer who runs a Discord bot that rewrites stack traces into plain English moved it from a paid LLM API to a free endpoint and documented five failures that emerged over two weeks. The migration exposed issues with hardcoded model names, rate-limit handling, token counting, and response parsing, leading the developer to build a wrapper for provider portability. The developer verified the issues were generic by reproducing two against a second endpoint.", "body_md": "My Discord bot has one job: when someone pastes a stack trace into our server's #help channel, it rewrites the trace into a plain-English explanation plus a likely fix. It ran for months on a paid LLM API at a few dollars a month — small money, but the kind that nags you when the workload is trivially bursty: silent for days, then thirty requests in an evening when a game update breaks everyone's mods.\n\nSo I pointed it at a free endpoint instead. The migration took an afternoon. The *consequences* of the migration took two weeks to fully shake out, because nothing failed loudly. Everything failed slightly. This is a log of the five things that broke, what each one taught me about provider portability, and the wrapper code that now sits between my bot and any LLM endpoint so I never have to debug these twice.\n\nThe endpoint I moved to is [MonkeyCode](https://monkeycode.dev), which offers free model access and a free server option — the right price shape for a hobby bot with spiky traffic. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Every failure below, though, is generic to switching OpenAI-compatible providers. I verified that by deliberately reproducing two of them against a second endpoint. Treat this as a portability field report, not a product review.\n\nMy config had the model name hardcoded in three places: the request builder, the logging line, and a cost-estimation function that divided tokens by a per-model price. When I swapped providers, I updated one of the three. The bot ran fine — and my logs confidently recorded the old model name for a week, which made every later debugging session confusing because I was reading telemetry for a model I wasn't using.\n\nFix: model identity is configuration, not code, and it flows from exactly one source.\n\n``` python\n# llm_config.py — single source of truth\nfrom dataclasses import dataclass\nimport os\n\n@dataclass(frozen=True)\nclass LLMConfig:\n    base_url: str\n    api_key: str\n    model: str          # whatever the CURRENT provider's catalog calls it\n    max_output_tokens: int\n    timeout_s: float\n\ndef load_config() -> LLMConfig:\n    return LLMConfig(\n        base_url=os.environ[\"LLM_BASE_URL\"],\n        api_key=os.environ[\"LLM_API_KEY\"],\n        model=os.environ[\"LLM_MODEL\"],\n        max_output_tokens=int(os.environ.get(\"LLM_MAX_OUTPUT\", \"600\")),\n        timeout_s=float(os.environ.get(\"LLM_TIMEOUT_S\", \"20\")),\n    )\n```\n\nBoring, obvious, and the root cause of the messiest week. A related trap: never assume a model name means the same thing across providers, or even across months on the same provider. Catalogs change. Check what your endpoint actually serves before writing it into an environment variable, and expect to change it again.\n\nOn the paid API, I had never once seen a 429. On a free tier, the evening burst pattern hit the limit within the first weekend. The truly bad part: my error handler treated non-200 responses as \"log and move on,\" so during peak hours the bot silently ignored half the stack traces posted. Users thought it was ignoring *them specifically* and reposted, which made the burst worse. A rate limit had become a feedback loop.\n\nFix: bounded retries with jittered backoff, and — critically — a visible degraded state.\n\n``` python\n# llm_call.py — retry wrapper with honest degradation\nimport asyncio\nimport random\nimport openai\n\nMAX_ATTEMPTS = 4\n\nasync def call_with_backoff(client, cfg, messages) -> str | None:\n    for attempt in range(MAX_ATTEMPTS):\n        try:\n            resp = await asyncio.to_thread(\n                client.chat.completions.create,\n                model=cfg.model,\n                messages=messages,\n                temperature=0.2,\n                max_tokens=cfg.max_output_tokens,\n                timeout=cfg.timeout_s,\n            )\n            return resp.choices[0].message.content\n        except openai.RateLimitError:\n            if attempt == MAX_ATTEMPTS - 1:\n                return None  # caller must say so out loud\n            base = min(2 ** attempt, 8)\n            await asyncio.sleep(base + random.uniform(0, base))\n        except openai.APITimeoutError:\n            return None  # a slow answer in a chat channel is a wrong answer\n    return None\n\nasync def explain_trace(client, cfg, trace: str) -> str:\n    result = await call_with_backoff(client, cfg, [\n        {\"role\": \"system\", \"content\": SYSTEM_PROMPT},\n        {\"role\": \"user\", \"content\": trace},\n    ])\n    if result is None:\n        return (\"⚠️ I'm rate-limited right now. Try again in a minute, \"\n                \"or trim the trace to the first error line — shorter \"\n                \"messages are more likely to get through.\")\n    return result\n```\n\nThe degraded message does two jobs: it resets user expectations, and it gives them an actionable workaround (shorter input). Silence is the worst possible response to a rate limit in an interactive product.\n\nThe old setup had headroom I never measured, so my prompt builder happily concatenated the system prompt, the full stack trace, and the last five channel messages for context. On the new model, long traces started returning explanations of *the wrong error* — because the actual exception line, at the end of the paste, had been truncated away by the context window.\n\nNo exception was raised. The model just answered a different question than the user asked. This is the nastiest class of provider-migration bug: output stays plausible while becoming wrong.\n\nFix: budget tokens explicitly and truncate deliberately, keeping the semantically important part. For stack traces, the important part is the *last* exception block, not the first frames:\n\n``` php\ndef trim_trace(trace: str, char_budget: int) -> str:\n    \"\"\"Keep the tail of the trace: the exception line matters most.\"\"\"\n    if len(trace) <= char_budget:\n        return trace\n    head_room = char_budget // 4\n    tail = trace[-(char_budget - head_room):]\n    # don't cut mid-line; align to a newline\n    nl = tail.find(\"\\n\")\n    if 0 < nl < 200:\n        tail = tail[nl + 1:]\n    return trace[:head_room] + \"\\n[...middle frames omitted...]\\n\" + tail\n```\n\nCharacter budgets are a crude proxy for tokens, but a 4:1 char-to-token estimate with a safety margin beats not thinking about it at all. The general principle: after any provider switch, find your longest realistic input and verify end-to-end what the model actually receives.\n\nThe bot edits its own Discord message progressively as tokens arrive — a nice touch that made it feel fast. The new endpoint delivered chunks on a different cadence: larger, less frequent batches. The visual result was a message that sat empty for seconds, then jumped. Worse, my progressive-edit logic issued a Discord API edit per chunk, and the chunkier cadence interacted badly with Discord's own rate limits on message edits.\n\nFix: I removed streaming for this use case. Deliberate regression, correct call. For a chat bot whose answers are a few hundred tokens, perceived speed comes more from a fast *first* reaction than from progressive rendering. The bot now posts \"🔎 reading the trace…\" immediately (a fast, guaranteed reaction), then replaces it with the full answer in one edit. Users rated it as feeling *faster* than the stuttering stream. The lesson generalizes: provider features that seem free — streaming, JSON modes, tool calling — are exactly where behavioral differences hide, because they're the least standardized parts of an API that is otherwise deliberately compatible.\n\nI had an uptime monitor hitting the endpoint's root URL. It stayed green through every incident above, because the server was always *up*; it was the model path that rate-limited, truncated, or lagged. A TCP-level health check for an LLM-backed feature is close to useless.\n\nFix: a synthetic probe that exercises the real path, run on a schedule, alerting on latency and content sanity:\n\n``` php\nasync def synthetic_probe(client, cfg) -> dict:\n    start = asyncio.get_event_loop().time()\n    result = await call_with_backoff(client, cfg, [\n        {\"role\": \"user\", \"content\":\n         \"Reply with exactly: PROBE_OK\"},\n    ])\n    elapsed = asyncio.get_event_loop().time() - start\n    return {\n        \"ok\": result is not None and \"PROBE_OK\" in result,\n        \"latency_s\": round(elapsed, 2),\n    }\n```\n\nTwo weeks of probe data also gave me something I never had on the paid API: an honest picture of the free tier's behavior under my real traffic pattern — when bursts collide with limits, what p50 latency actually feels like. If you run one thing from this article, run the probe. It converts \"the free tier seems flaky\" into data you can act on.\n\nThe switch was worth it: the bot's operating cost for this workload went to zero, and every failure above was a latent bug in *my* code that the paid tier's headroom had been papering over. Free capacity is unforgiving in a useful way.\n\nBut be honest about the fit:\n\nIf you want a concrete place to try this, MonkeyCode's free model access and free server option are what this bot currently runs on; the wrapper above means it could run somewhere else tomorrow, which is precisely the property worth engineering for. The migration took an afternoon. The engineering to make the next migration boring took two weeks — and it's the part that was actually worth doing.", "url": "https://wpnews.pro/news/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke", "canonical_source": "https://dev.to/codepy_1473/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke-2ik5", "published_at": "2026-08-12 23:16:05+00:00", "updated_at": "2026-08-12 23:45:55.987133+00:00", "lang": "en", "topics": ["large-language-models", "developer-tools", "ai-products"], "entities": ["MonkeyCode", "Discord", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke", "markdown": "https://wpnews.pro/news/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke.md", "text": "https://wpnews.pro/news/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke.txt", "jsonld": "https://wpnews.pro/news/i-moved-my-discord-bot-off-a-paid-llm-api-five-things-broke.jsonld"}}