This picture summarizes it well:
Here on my blog, for this popular blog post I get a lot of comments. 28k blog comments over the years. Some of them are terribly written and hard to understand, so I let AI suggest a rewrite. That code that sends the blog post comment to AI, I actually fire off three times: once with OpenAI gpt-5
, once with OpenAI gpt-5-mini
, and once with Claude claude-opus-4.8
. I use my human eyes and judgement to evaluate the results, and I can tell you they do equally well. Only the slightest differences.
The surprising thing is how amazingly slow OpenAI's gpt-5
is! It's nearly 10x slower than claude-opus-4.8
. What's up with that!?
It's also clear that the latency difference between gpt-5-mini
and gpt-5
is significant. At the time of writing, the input token price difference between gpt-5.4
and gpt-5.4-mini
is $2.50 compared to $0.75! That's a 3x difference.
If you're constructing a prompt the API, use Claude.
If you have to use OpenAI, consider the
mini
model because it'sbothcheaperandfaster.
Before I added Claude, I used to use litellm
to wrap OpenAI's models. The code looks like this:
response = litellm.completion(
model="openai-gpt-5",
api_key=settings.OPENAI_API_KEY,
messages=my_prompt_messages,
)
Unlike, if you use the native OpenAI Python SDK the invocation looks like this:
client = openai.OpenAI(api_key=settings.OPENAI_API_KEY)
response = client.responses.create(
model="gpt-5,
input=my_prompt_messages,
)
I measured the difference, in speed, where I compare using the OpenAI SDK versus the litellm
wrapper and the difference looks like this:
Granted, in June I "only" did a bit over 30 of these calls, but strangely there's a difference!
I don't have the intricate knowledge to understand why the litellm
makes the total time different from using the SDK. (not sure I care either!)
Either way, I'm moving away from litellm
and only use the SDKs provided by Claude and OpenAI. Feels safer given the CVEs we've seen this year on litellm
.