{"slug": "same-model-three-apis-contract-testing-with-docker-model-runner", "title": "Same Model, Three APIs: Contract-Testing with Docker Model Runner", "summary": "A developer demonstrated a contract-testing approach that runs OpenAI-, Anthropic-, and Ollama-compatible protocol adapters against a single local model environment using Docker Model Runner, arguing that \"OpenAI-compatible\" is shorthand rather than a portability guarantee. The method normalizes each wire format into a small ChatResult type, then applies shared tests for bounded responses, streaming deltas, cancellation, error mapping, JSON schema validation, and tool calling. Unsupported behavior is made explicit through an AdapterCapabilities type so applications fail fast instead of discovering gaps mid-run.", "body_md": "“OpenAI-compatible” is useful shorthand. It is not a portability guarantee.\n\nTwo APIs may accept similar chat messages while differing in streaming events, tool calls, token counts, error shapes, stop reasons, and supported parameters. Swapping a base URL can make a demo work while leaving the production abstraction full of assumptions.\n\nDocker Model Runner is a useful local contract target because it exposes OpenAI-, Anthropic-, and Ollama-compatible formats over one local model environment. Instead of comparing three remote providers, we can test three protocol adapters against controlled local infrastructure.\n\nKeep provider objects out of domain code:\n\n```\ntype ChatRequest = {\n  messages: Array<{ role: \"user\" | \"assistant\"; content: string }>;\n  maxOutputTokens: number;\n  stream?: boolean;\n};\n\ntype ChatResult = {\n  text: string;\n  finishReason: \"stop\" | \"length\" | \"tool\" | \"unknown\";\n  inputTokens?: number;\n  outputTokens?: number;\n};\n\ninterface ModelAdapter {\n  id: string;\n  chat(request: ChatRequest): Promise<ChatResult>;\n}\n```\n\nEach adapter translates its wire format into this deliberately small result. Do not force unsupported provider details into fake equivalence. Optional usage fields are more honest than invented token counts.\n\nFor host processes, Docker's documented base URLs are:\n\n| Client shape | Base URL | \n|---|---|\n| OpenAI-compatible | `http://localhost:12434/engines/v1` | \n| Anthropic-compatible | `http://localhost:12434` | \n| Ollama-compatible | `http://localhost:12434` | \n\nFrom containers, use `model-runner.docker.internal` as documented for your Docker environment. Docker Engine setups may require an `extra_hosts` host-gateway mapping.\n\nCreate the matrix once:\n\n``` js\nconst adapters: ModelAdapter[] = [\n  openAiAdapter({ baseURL: \"http://localhost:12434/engines/v1\" }),\n  anthropicAdapter({ baseURL: \"http://localhost:12434\" }),\n  ollamaAdapter({ baseURL: \"http://localhost:12434\" }),\n];\n```\n\nAll three paths can reach the same configured model, but generated text may still vary because request translation and sampling surfaces differ. Contract-test properties your application actually needs:\n\n``` js\ndescribe.each(adapters)(\"$id adapter\", (adapter) => {\n  it(\"returns a bounded non-empty response\", async () => {\n    const result = await adapter.chat({\n      messages: [{ role: \"user\", content: \"Reply with one color.\" }],\n      maxOutputTokens: 16,\n    });\n\n    expect(result.text.trim()).not.toBe(\"\");\n    expect(result.text.length).toBeLessThan(200);\n    expect([\"stop\", \"length\", \"tool\", \"unknown\"])\n      .toContain(result.finishReason);\n  });\n});\n```\n\nThen add separate contracts for the risky seams.\n\nConcatenate provider-specific delta events and require the same normalized terminal result as the non-streaming path. Also test cancellation and an interrupted stream.\n\nSend an invalid request and map each wire error into an application category such as `invalid_request`, `unavailable`, or `rate_limited`. Preserve the original error as diagnostic evidence, not as control flow.\n\nIf the application depends on JSON output, validate the parsed result against your own schema. A protocol accepting `response_format` does not remove semantic validation.\n\nRun tool contracts only for a model and inference engine that support them. Docker documents OpenAI-compatible function calling for compatible models under llama.cpp; a red test on an unsupported model is not evidence of a broken adapter.\n\nPortability improves when unsupported behavior is explicit:\n\n```\ntype AdapterCapabilities = {\n  streaming: boolean;\n  tools: boolean;\n  jsonMode: boolean;\n  tokenUsage: \"reported\" | \"estimated\" | \"unavailable\";\n};\n```\n\nThe application can then fail fast instead of discovering halfway through a run that the selected adapter cannot satisfy a required feature.\n\nVersion the capability observation rather than treating it as eternal configuration:\n\n```\ntype TestedAdapter = {\n  adapterVersion: string;\n  protocol: \"openai\" | \"anthropic\" | \"ollama\";\n  modelId: string;\n  engine: string;\n  observedAt: string;\n  capabilities: AdapterCapabilities;\n};\n```\n\nThe same protocol adapter can behave differently with another engine or model. Cache the manifest only for the exact tuple you tested, and rerun the matrix when any member changes.\n\nInclude negative tests, not only successful prompts. Abort a stream, send an unsupported parameter, exceed the configured context, and request a tool from a model without tool support. A portable adapter should translate each response into an explicit application error while preserving enough provider detail for diagnosis. Silent parameter dropping is a failed contract even when text still arrives.\n\nDocker Model Runner itself documents important differences. Its local OpenAI-compatible endpoint does not require an API key and ignores the authorization header. Token counting uses the model's native encoder and can differ from OpenAI. Supported features depend on the engine and model.\n\nThose are exactly the reasons to contract-test rather than infer compatibility from the URL shape.\n\nA compatible endpoint lowers migration cost. A provider abstraction becomes trustworthy only after you test the behaviors on which your application depends: stream completion, errors, tool arguments, structured output, cancellation, and usage accounting.\n\nRunning that matrix locally makes it fast and inexpensive. More importantly, it reveals where the abstraction is genuinely portable—and where the provider-specific capability must remain visible.\n\nUse local compatibility to improve your adapter, not to erase provider identity. Production policies may still differ for credentials, data residency, rate limits, safety controls, and model lifecycle even when the request and response shapes look familiar.", "url": "https://wpnews.pro/news/same-model-three-apis-contract-testing-with-docker-model-runner", "canonical_source": "https://dev.to/raju_dandigam/same-model-three-apis-contract-testing-with-docker-model-runner-3d67", "published_at": "2026-09-22 16:49:21+00:00", "updated_at": "2026-09-22 16:52:58.704556+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-infrastructure", "mlops", "agent-protocols"], "entities": ["Docker Model Runner", "Docker", "OpenAI", "Anthropic", "Ollama", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/same-model-three-apis-contract-testing-with-docker-model-runner", "markdown": "https://wpnews.pro/news/same-model-three-apis-contract-testing-with-docker-model-runner.md", "text": "https://wpnews.pro/news/same-model-three-apis-contract-testing-with-docker-model-runner.txt", "jsonld": "https://wpnews.pro/news/same-model-three-apis-contract-testing-with-docker-model-runner.jsonld"}}