You install the OpenAI Python package.
You import OpenAI
.
You call client.chat.completions.create()
.
And Claude answers.
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["ANTHROPIC_API_KEY"],
base_url="https://api.anthropic.com/v1/",
)
response = client.chat.completions.create(
model="claude-sonnet-4-6",
messages=[
{"role": "user", "content": "Explain this in one sentence."}
],
)
print(response.choices[0].message.content)
That looks wrong the first time you see it.
If the SDK says openai
, shouldn't an OpenAI model answer?
No—and the reason matters far beyond this one code sample.
Three independent choices are hiding in that snippet:
The package name is not the model name.
In the example above:
OpenAI Python SDK
↓ speaks OpenAI-style HTTP
Anthropic compatibility endpoint
↓ maps the request
Claude Sonnet
↓ result is mapped back
OpenAI-shaped response
↓ parsed by the SDK
response.choices[0].message.content
Anthropic officially provides this compatibility layer for testing and comparing Claude with existing OpenAI integrations. It is not the recommended production path for most Claude-first applications, for reasons we will get to shortly.
But first, let us separate the layers developers accidentally compress into the word “AI.”
The useful mental model is not “SDK → model.” It is:
┌───────────────────────────────────────────────┐
│ 1. Application + SDK │
│ Builds a request and parses a response │
├───────────────────────────────────────────────┤
│ 2. API endpoint / gateway │
│ Auth, routing, policy, protocol mapping │
├───────────────────────────────────────────────┤
│ 3. Serving layer │
│ Scheduling, batching, streaming, metrics │
├───────────────────────────────────────────────┤
│ 4. Inference runtime + model │
│ Prompt → tokens → generation → tokens │
└───────────────────────────────────────────────┘
These layers may live in one program, several containers, or multiple companies' infrastructure. The boundaries are conceptual, but the responsibilities are different.
An SDK usually gives you:
Without an SDK, the same job can be done with an HTTP client:
import os
import httpx
response = httpx.post(
"https://api.anthropic.com/v1/messages",
headers={
"x-api-key": os.environ["ANTHROPIC_API_KEY"],
"anthropic-version": "2023-06-01",
"content-type": "application/json",
},
json={
"model": "claude-sonnet-4-6",
"max_tokens": 100,
"messages": [{"role": "user", "content": "Hello"}],
},
)
data = response.json()
print(data["content"][0]["text"])
The native Anthropic response uses content[]
. The OpenAI-style response uses choices[]
.
Those shapes are API contracts. Neither is the natural output format of a neural network.
Consider these clients:
OpenAI(
api_key=os.environ["OPENAI_API_KEY"],
base_url="https://api.openai.com/v1",
)
OpenAI(
api_key=os.environ["ANTHROPIC_API_KEY"],
base_url="https://api.anthropic.com/v1/",
)
OpenAI(
api_key="ollama", # required by the client; ignored locally
base_url="http://localhost:11434/v1/",
)
The calling style barely changes, but the request crosses three completely different trust, billing, latency, and data boundaries.
The base_url
can point to:
This is why “we use the OpenAI SDK” tells an architect almost nothing about where prompts go.
The next questions should be:
base_url
?At the lowest useful level, a language model works with numbers.
The prompt is formatted and tokenized. The model produces logits. A generation strategy selects new token IDs. Those IDs are decoded into text.
For example, Hugging Face Transformers documents that generate()
returns token sequences—or a richer internal ModelOutput
when requested:
generated_ids = model.generate(**model_inputs, max_new_tokens=50)
text = tokenizer.batch_decode(
generated_ids,
skip_special_tokens=True,
)[0]
There is no universal neural-network law requiring the result to contain:
{
"choices": [],
"finish_reason": "stop",
"usage": {
"prompt_tokens": 10,
"completion_tokens": 5
}
}
That public JSON shape is assembled by software around the generation runtime.
A serving system may also produce internal text, token IDs, finish metadata, timing information, cache statistics, and scheduler state. The important distinction is not “the engine only returns text.” The important distinction is:
Provider JSON is a network contract created by the serving/API layer—not an intrinsic property of the model weights.
Projects such as vLLM make this visible: the inference machinery returns internal request outputs, while the OpenAI-compatible server exposes endpoints with OpenAI-style schemas.
The model
field is a request to the service, not a Python import.
The endpoint decides how to interpret it.
response = client.chat.completions.create(
model="anthropic/claude-sonnet-4.6",
messages=[{"role": "user", "content": "Hello"}],
)
A gateway might:
So even the model string is not always the complete deployment identity. In production, log the resolved provider, model version, request ID, and routing decision whenever the platform exposes them.
This is where a convenient prototype becomes a quiet production bug.
Two services can support /v1/chat/completions
and still disagree on:
Anthropic documents several concrete limitations in its OpenAI compatibility layer:
strict
for function calling is ignored;response_format
, logprobs
, and several other fields are ignored;Some unsupported fields are silently ignored.
That last behavior is more dangerous than a clean error. Your code can compile, your request can return 200
, and your assumption can still be false.
Ollama uses equally careful wording: it supports parts of the OpenAI API. vLLM documents its own list of supported and additional parameters.
Compatibility is a spectrum, not a boolean.
Suppose this prototype works:
client = OpenAI(
api_key=os.environ["GATEWAY_API_KEY"],
base_url=os.environ["LLM_BASE_URL"],
)
Changing an environment variable may be enough for basic chat completion.
Then production adds:
Now the lowest common denominator begins to cost you.
The abstraction was not free. You deferred the translation work to a gateway—and accepted its fidelity limits.
| Situation | Sensible default | Main trade-off |
|---|---|---|
| Claude-first production app | Native Anthropic SDK | Best Claude feature coverage; tighter vendor coupling |
| OpenAI-first production app | Native OpenAI SDK | Best OpenAI feature coverage; tighter vendor coupling |
| Model evaluation | OpenAI-compatible surface or gateway | Fast switching; feature comparisons may be incomplete |
| Mostly portable text generation | Common gateway contract | Simple integration; lowest-common-denominator risk |
| Heavy tools, streaming, or multimodal use | Native provider adapters behind your own interface | More code; explicit and testable behavior |
| Local development | Ollama or vLLM compatibility endpoint | Convenient; confirm exactly which features are supported |
For a serious multi-provider application, I prefer a small internal interface whose implementation uses native provider SDKs.
For example:
from typing import Protocol
class TextModel(Protocol):
def generate(self, prompt: str) -> str: ...
class ClaudeModel:
def generate(self, prompt: str) -> str:
...
class OpenAIModel:
def generate(self, prompt: str) -> str:
...
This is more work than changing base_url
, but it makes the lossy parts visible. Your domain code depends on your contract, while provider-specific capabilities remain available inside each adapter.
If you choose a universal gateway instead, create contract tests for every feature you rely on:
✓ plain text
✓ streaming
✓ tool call arguments
✓ strict structured output
✓ image input
✓ stop reasons
✓ usage accounting
✓ retryable error classification
Do not test only whether the first “Hello” request succeeds.
When someone says:
“We use the OpenAI SDK.”
Translate it to:
“This code uses a client designed around an OpenAI API shape.”
It does not automatically mean:
Remember the chain:
SDK → API contract → endpoint/router → serving runtime → model
The SDK speaks a protocol.
The endpoint receives and may route the request.
The serving stack runs—or delegates—the generation work.
The model generates token probabilities.
Once those responsibilities are separate in your head, “OpenAI SDK calling Claude” stops looking like magic. It becomes what it always was: one client speaking a compatible network contract to an endpoint that knows how to reach Claude.
What broke first when you switched LLM providers: streaming, tool calls, structured output, or usage accounting?
Share the failure mode in the comments. Those edge cases are where “compatible” gets interesting.
Disclosure: This article is based on the author's original technical material and subject-matter knowledge. AI was used to help restructure the article, edit the English, and create the cover image. The technical claims and cited sources were reviewed before publication.