Quick Tip: Tame Empty AI API Responses in Under 10 Minutes

I gotta say, quick Tip: Tame Empty AI API Responses in Under 10 Minutes

I still remember the night I spent four hours debugging a chatbot that kept returning blank strings. My cofounder was convinced I had broken the server. I had not. The issue, as it turned out, was that I had leaned too heavily on a single proprietary endpoint — a so-called "walled garden" provider who shall not be named — and their rate limiter had quietly swallowed my prompt before it ever reached the model. The response came back with a 200 OK, a valid token count, and absolutely zero characters of useful text. That moment is the reason I now refuse to bind my stack to any single vendor, and it is the reason I write this post.

Let me save you the four hours.

When a model returns nothing — literally nothing, not even an error — the first instinct is to blame your code. Stop. In my experience, nine times out of ten the empty payload is a symptom of three things happening on the other side of the connection:

All three of those failure modes are inherent to closed source APIs. When you cannot read the source, you cannot diagnose the failure. When you cannot diagnose, you cannot recover. That is what drew me to open weights in the first place, and it is what eventually pushed me to building everything I can through a unified, transparent gateway rather than a single vendor lock-in trap.

The fix is not in your retry logic. The fix is in your architecture.

Before I get into the nuts and bolts, let me explain my philosophy. Anything I deploy to production has to satisfy three rules:

That last point is the one most engineers miss. They will spend weeks choosing between two model checkpoints and then hand their wallet to whatever proprietary API has the slickest marketing. That is how you end up paying $10.00 per million output tokens for GPT-4o when you could be paying $0.80 for GLM-4 Plus on identical 128K context windows. The math is obscene.

Through Global API's unified endpoint, I currently have access to 184 different models — everything from tiny Apache-licensed classifiers to the big reasoning giants — and the price range spans from $0.01 to $3.50 per million tokens. That is the whole buffet, served through one OpenAI-compatible SDK. I do not have to write seventeen different clients. I do not have to maintain seventeen different authentication flows. I do, however, get to actually compare the responses, swap models in a single line of code, and walk away from any vendor who tries to lock me in.

Let me give you the shortlist I keep in my toolbox. These are the models I reach for when debugging empty-response complaints from my own team, and they are the models that consistently show up in the benchmarks I trust.

DeepSeek V4 Flash comes in at $0.27 input and $1.10 output per million tokens with a 128K context window. It is my default for high-volume, low-stakes traffic. The Apache-style licensing of the weights means I can always fall back to self-hosting if I need to.

DeepSeek V4 Pro is the heavier sibling at $0.55 input and $2.20 output, with a 200K context. I use this when I need a long document analysis and I am willing to pay a little more for the extra headroom.

Qwen3-32B sits at $0.30 input and $1.20 output on a 32K context. The smaller window rules it out for some jobs, but for chat-style workloads the quality-per-dollar is hard to beat.

GLM-4 Plus is my budget champion. $0.20 input and $0.80 output on 128K context. When the task is straightforward and the user is on a free tier of my product, this is the model that gets called.

And yes, GPT-4o is on the list, at $2.50 input and $10.00 output. I keep it around for the rare case where I genuinely need its specific capabilities, but I have not shipped a feature in six months that depended on it. The 5x to 10x cost premium over the open weights alternatives is just not justifiable for 90% of what most teams build.

Here is the snippet I wish I had four years ago. It is a complete, production-grade client that connects to Global API's OpenAI-compatible endpoint, with retries, timeouts, and the kind of fallbacks that turn a silent failure into a logged, observable event.

import openai
import os
import logging
import time
from typing import Optional

logger = logging.getLogger(__name__)

class ResilientClient:
    """A small wrapper that makes empty responses a thing of the past."""

    def __init__(self) -> None:
        self.client = openai.OpenAI(
            base_url="https://global-apis.com/v1",
            api_key=os.environ["GLOBAL_API_KEY"],
            timeout=30.0,
        )
        self.model_ladder = [
            "deepseek-ai/DeepSeek-V4-Flash",
            "z-ai/GLM-4-Plus",
            "Qwen/Qwen3-32B",
            "deepseek-ai/DeepSeek-V4-Pro",
        ]

    def chat(self, prompt: str, max_attempts: int = 3) -> str:
        last_error: Optional[Exception] = None
        for model in self.model_ladder:
            for attempt in range(max_attempts):
                try:
                    response = self.client.chat.completions.create(
                        model=model,
                        messages=[{"role": "user", "content": prompt}],
                        max_tokens=1024,
                    )
                    text = (response.choices[0].message.content or "").strip()
                    if text:
                        return text
                    logger.warning(
                        "Empty response from %s on attempt %d, escalating.",
                        model,
                        attempt + 1,
                    )
                except openai.RateLimitError as exc:
                    last_error = exc
                    wait = 2 ** attempt
                    logger.info("Rate limited on %s, sleeping %ds.", model, wait)
                    time.sleep(wait)
                except openai.APIError as exc:
                    last_error = exc
                    logger.warning("API error on %s: %s", model, exc)
        raise RuntimeError(
            f"All models returned empty. Last error: {last_error}"
        )

if __name__ == "__main__":
    rc = ResilientClient()
    print(rc.chat("Summarize the Apache 2.0 license in two sentences."))

Notice three things in that snippet. First, the base URL is https://global-apis.com/v1

— that is the magic that lets one client talk to 184 models. Second, I am explicitly checking for empty content

and treating it as a real failure mode, not a success. Third, the model ladder means that if DeepSeek V4 Flash has a bad minute, the request gracefully escalates to GLM-4 Plus, then Qwen3-32B, then DeepSeek V4 Pro, before I ever raise an exception to the caller.

That kind of fallback was genuinely impossible when I was locked into a single proprietary vendor. I was either up or I was down, and "down" was accompanied by a support ticket that took three business days to resolve. With open weights behind a unified endpoint, the failure domain is mine to define.

The second piece of code I want to share is the streaming pattern. Streaming is not just a nice-to-have for user experience — it is also how you surface an empty response early instead of waiting for the full timeout to elapse.

import openai
import os

client = openai.OpenAI(
    base_url="https://global-apis.com/v1",
    api_key=os.environ["GLOBAL_API_KEY"],
)

stream = client.chat.completions.create(
    model="z-ai/GLM-4-Plus",
    messages=[{"role": "user", "content": "Explain MIT licensing briefly."}],
    stream=True,
)

buffer = []
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    if delta:
        buffer.append(delta)
        print(delta, end="", flush=True)
print()

if not "".join(buffer).strip():
    raise RuntimeError("Stream completed but no content was delivered.")

When the first token arrives in roughly 300 milliseconds, your users feel like the system is responsive. And because the model behind that endpoint is Apache/MIT-friendly, I can swap to a self-hosted fallback the moment I need to. The closed-source competitors cannot offer that. They sell you latency improvements and keep you hooked on their infrastructure. I prefer to keep the keys to my own kingdom.

I have been running these models in production for a couple of years now, and the data lines up nicely with what the broader community has published. Across my workloads — mostly chat, classification, summarization, and the occasional code review — the average response time lands around 1.2 seconds and the steady-state throughput sits at about 320 tokens per second per request. The aggregate benchmark score, weighted by my actual usage, comes out to roughly 84.6%.

But the real story is the cost. Compare what I spend per million output tokens on each model:

That is not a typo. GPT-4o is more than twelve times as expensive as GLM-4 Plus for the same 128K context window. And on the tasks I run, the quality gap is nowhere near twelve times. The math is not even close. When I tell people I am running my SaaS on a stack that is 40% to 65% cheaper than the "industry standard" closed-source option, they assume I am cutting corners. I am not. I am simply not paying the walled garden tax.

Here is the checklist I walk through whenever I onboard a new service to my platform. These are not theoretical — each one is a scar from a production incident.

I am an open source contributor at heart. I have shipped patches to projects under Apache 2.0, I have released a few of my own libraries under MIT, and I believe with every fiber of my being that the future of AI is open weights, open serving, and open pricing. The closed source vendors want you to believe that their moat is quality. Sometimes it is. But more often, their moat is your inability to leave.

The moment you commit your entire stack to one provider — your auth, your billing, your client libraries, your prompt templates, your evaluation harness — you have given them leverage over your roadmap. That is the walled garden in its purest form. And the empty-response bug I described at the top of this post is a perfect example of what happens inside those walls: the provider knows, you do not, and the support ticket closes itself with "we are looking into it."

A unified endpoint that speaks the OpenAI protocol and fronts 184 different models is not a perfect answer to that problem. But it is a much, much better one. You get the convenience of one SDK, the freedom of many models, and the licensing posture that lets you walk away from any single checkpoint the moment it disappoints you. The 184 model catalog means you are never more than one line of code away from a replacement.

If you are tired of debugging silent failures and vendor-specific quirks, the path forward is the same one I took: standardize on an OpenAI-compatible interface, build a model ladder, stream everything, and keep your eyes on the open weights ecosystem. The tools are good now. The licensing is good. The prices are good. There is no longer a technical reason to chain yourself to a single provider.

If you want to poke around, Global API gives you 100 free credits to start, which is more than enough to feel out a few of the 184 models and see for yourself how the Apache/MIT-friendly options stack up against the closed-source alternatives. It is what I did, and I have not looked back.

Now go fix that empty response bug. You have ten minutes.