Same Model, Three APIs: Contract-Testing with Docker Model Runner

A developer demonstrated a contract-testing approach that runs OpenAI-, Anthropic-, and Ollama-compatible protocol adapters against a single local model environment using Docker Model Runner, arguing that "OpenAI-compatible" is shorthand rather than a portability guarantee. The method normalizes each wire format into a small ChatResult type, then applies shared tests for bounded responses, streaming deltas, cancellation, error mapping, JSON schema validation, and tool calling. Unsupported behavior is made explicit through an AdapterCapabilities type so applications fail fast instead of discovering gaps mid-run.

“OpenAI-compatible” is useful shorthand. It is not a portability guarantee. Two APIs may accept similar chat messages while differing in streaming events, tool calls, token counts, error shapes, stop reasons, and supported parameters. Swapping a base URL can make a demo work while leaving the production abstraction full of assumptions. Docker Model Runner is a useful local contract target because it exposes OpenAI-, Anthropic-, and Ollama-compatible formats over one local model environment. Instead of comparing three remote providers, we can test three protocol adapters against controlled local infrastructure. Keep provider objects out of domain code: type ChatRequest = { messages: Array<{ role: "user" | "assistant"; content: string } ; maxOutputTokens: number; stream?: boolean; }; type ChatResult = { text: string; finishReason: "stop" | "length" | "tool" | "unknown"; inputTokens?: number; outputTokens?: number; }; interface ModelAdapter { id: string; chat request: ChatRequest : Promise<ChatResult ; } Each adapter translates its wire format into this deliberately small result. Do not force unsupported provider details into fake equivalence. Optional usage fields are more honest than invented token counts. For host processes, Docker's documented base URLs are: | Client shape | Base URL | |---|---| | OpenAI-compatible | http://localhost:12434/engines/v1 | | Anthropic-compatible | http://localhost:12434 | | Ollama-compatible | http://localhost:12434 | From containers, use model-runner.docker.internal as documented for your Docker environment. Docker Engine setups may require an extra hosts host-gateway mapping. Create the matrix once: js const adapters: ModelAdapter = openAiAdapter { baseURL: "http://localhost:12434/engines/v1" } , anthropicAdapter { baseURL: "http://localhost:12434" } , ollamaAdapter { baseURL: "http://localhost:12434" } , ; All three paths can reach the same configured model, but generated text may still vary because request translation and sampling surfaces differ. Contract-test properties your application actually needs: js describe.each adapters "$id adapter", adapter = { it "returns a bounded non-empty response", async = { const result = await adapter.chat { messages: { role: "user", content: "Reply with one color." } , maxOutputTokens: 16, } ; expect result.text.trim .not.toBe "" ; expect result.text.length .toBeLessThan 200 ; expect "stop", "length", "tool", "unknown" .toContain result.finishReason ; } ; } ; Then add separate contracts for the risky seams. Concatenate provider-specific delta events and require the same normalized terminal result as the non-streaming path. Also test cancellation and an interrupted stream. Send an invalid request and map each wire error into an application category such as invalid request , unavailable , or rate limited . Preserve the original error as diagnostic evidence, not as control flow. If the application depends on JSON output, validate the parsed result against your own schema. A protocol accepting response format does not remove semantic validation. Run tool contracts only for a model and inference engine that support them. Docker documents OpenAI-compatible function calling for compatible models under llama.cpp; a red test on an unsupported model is not evidence of a broken adapter. Portability improves when unsupported behavior is explicit: type AdapterCapabilities = { streaming: boolean; tools: boolean; jsonMode: boolean; tokenUsage: "reported" | "estimated" | "unavailable"; }; The application can then fail fast instead of discovering halfway through a run that the selected adapter cannot satisfy a required feature. Version the capability observation rather than treating it as eternal configuration: type TestedAdapter = { adapterVersion: string; protocol: "openai" | "anthropic" | "ollama"; modelId: string; engine: string; observedAt: string; capabilities: AdapterCapabilities; }; The same protocol adapter can behave differently with another engine or model. Cache the manifest only for the exact tuple you tested, and rerun the matrix when any member changes. Include negative tests, not only successful prompts. Abort a stream, send an unsupported parameter, exceed the configured context, and request a tool from a model without tool support. A portable adapter should translate each response into an explicit application error while preserving enough provider detail for diagnosis. Silent parameter dropping is a failed contract even when text still arrives. Docker Model Runner itself documents important differences. Its local OpenAI-compatible endpoint does not require an API key and ignores the authorization header. Token counting uses the model's native encoder and can differ from OpenAI. Supported features depend on the engine and model. Those are exactly the reasons to contract-test rather than infer compatibility from the URL shape. A compatible endpoint lowers migration cost. A provider abstraction becomes trustworthy only after you test the behaviors on which your application depends: stream completion, errors, tool arguments, structured output, cancellation, and usage accounting. Running that matrix locally makes it fast and inexpensive. More importantly, it reveals where the abstraction is genuinely portable—and where the provider-specific capability must remain visible. Use local compatibility to improve your adapter, not to erase provider identity. Production policies may still differ for credentials, data residency, rate limits, safety controls, and model lifecycle even when the request and response shapes look familiar.