{"slug": "your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly", "title": "Your gateway's public model list is a contract. Most of them break it quietly.", "summary": "A developer spent a day calling the public model-list endpoints of four LLM gateways twice, twenty minutes apart, and compared the responses against vendor documentation, finding that these lists change silently and are treated as contracts they do not honor. The same public endpoint returned 477 models on 22 September and 199 models on 29 September, while OpenRouter's list went from 445 to 460 models in ten days, and one gateway's response schema dropped the `capabilities` field entirely, silently breaking every derived metric built on it. The developer recommends dating any figure pulled from an endpoint and using optional chaining such as `capabilities?.tool_calling` so a missing key does not crash clients.", "body_md": "Every LLM gateway publishes an endpoint that lists what you can call. Clients read it,\n\npricing pages quote it, comparison articles count it, and nobody treats it as an interface\n\nthat can change. That is the mistake. I spent a day calling four of them and comparing what\n\ncame back against what the vendors' own documentation says.\n\nOn 29 September 2026 I called the public model list of each gateway, twice, twenty minutes\n\napart, with no API key, and compared the response against whatever the vendor documents\n\nabout it. Nothing here needed an account, which is the point — a list that requires a key\n\nto read is not a public list.\n\nFive things, in the order they tend to bite:\n\nThis is the cheapest check and the one that fails most often, because it costs nothing to\n\nverify and almost nobody does.\n\nAn OpenAI-compatible client appends its own path. The Python SDK takes\n\n`base_url=\"https://example.com/v1\"` and then calls `/chat/completions` on top of\n\nit; the `OPENAI_BASE_URL` environment variable behaves the same way, which the\n\n[openai-python README](https://github.com/openai/openai-python) documents. An Anthropic\n\nclient appends `/v1/messages` to whatever base you hand it, as the\n\n[Messages API reference](https://docs.anthropic.com/en/api/messages) shows.\n\nSo the base URL is half of a path, and if the vendor writes the wrong half the client gets\n\na 404 that reads like an auth problem. On my own site I had this backwards in one place and\n\nright in another: the OpenAI base was correct on the SDK page and wrong in the file written\n\nspecifically for agents to read, because both were generated from a single JSON registry and\n\nthe stale value sat in that registry. One wrong entry, two surfaces, one of them broken.\n\nThe test takes one request, and the status code is the whole answer:\n\n```\ncurl -s -o /dev/null -w '%{http_code}\n' -X POST https://example.com/v1/chat/completions\n```\n\nA `401` means the route exists and wants a key. A `404` means the route does not exist, and\n\nno key will help. That distinction is the entire test. Run it against the Anthropic base too\n\n(`POST <base>/v1/messages`), because the two dialects are usually mounted at different\n\nprefixes and only one of them is documented carefully.\n\nModel counts move. Mine moved a lot: the same public endpoint returned 477 models on 22\n\nSeptember and 199 models on 29 September. OpenRouter's public list returned 460 models when I\n\ncalled it on 29 September, against 445 models in a snapshot taken ten days earlier. Neither number\n\nis a lie. Both are dated observations that somebody froze into a sentence and walked away\n\nfrom.\n\nThe consequence is narrower than it sounds and more annoying: a page that says it serves 477\n\nmodels gets checked by a reader who sees a different number, and the reader concludes the\n\npage is inventing things. The fix is not to update the number more often. It is to stop\n\nputting a moving number in a sentence with no date on it. Any figure that comes from an\n\nendpoint should carry the date it was read and the exact endpoint it came from, so a reader\n\ncan reproduce or refute it in one request.\n\nThis is the one that costs real time and that no checklist covers.\n\nA model list is not a fixed schema. In the case I hit, a response carrying `id`,\n\n`context_length`, `max_output_tokens`, `capabilities` and `owned_by` came back a week later\n\nwith `id`, `object`, `created`, `owned_by`, `tier` and `modality`, and no\n\n`capabilities` at all. Every derived number built on those fields silently became\n\nuncomputable. Shares of models supporting tool calling, shares accepting image input, counts\n\nof combination routes: all computed from a field that had stopped being returned. The pages\n\nkept rendering, the percentages kept being quoted, and nothing failed, because nothing\n\nchecked.\n\nIf you build against one of these lists, the fields you read are the contract:\n\n`capabilities?.tool_calling` survives a missing key.\n`capabilities.tool_calling` is a crash waiting for a release.\n[OpenRouter's FAQ](https://openrouter.ai/docs/faq) states plainly that it charges a fee when\n\nyou purchase credits, and that free-model rate limits are determined by how much credit you\n\nhave bought. The specific figures, the percentage and the minimum and the requests per day,\n\nare filled in at render time, so a plain HTTP fetch returns those sentences with the numbers\n\nmissing. The prose is the contract. The numbers are configuration.\n\nThat is not a criticism, it is a warning about how gateways get compared. If you are writing\n\nthe comparison, link the sentence you can actually cite and state the figure as a dated\n\nobservation. If you are picking a gateway, a number in a blog post from three months ago is\n\nnot a specification, and it was probably read out of the same bundle you could have read\n\nyourself.\n\nThe most useful thing a gateway model list can tell you is which entries are safe to pin and\n\nwhich are aliases that move underneath you. Almost none of them say. A route named\n\n`auto/best-coding` is a policy, not a model, and pinning it in production is a different\n\ndecision from pinning a specific model id. If the response does not distinguish the two, you\n\nfind out by shipping.\n\nAll of the above collapses into five requests you can run before pointing production at\n\nanything:\n\n`POST <base>/chat/completions` and `POST <base>/v1/messages`. Expect 401, not 404.` GET <base>/models` twice, twenty minutes apart. If the count or the field set moves,\nyou have learned something the README will not tell you.\nNone of this needs a key, an account, or anyone's permission. It takes 10 minutes, and it\n\nis the difference between integrating with a gateway and betting on one.\n\n*I work on [FreeModel by Aiglade](https://freemodel.online/), an AI model gateway, and the\n477-to-199 case in section three is my own endpoint. It is why I ran these checks on\neveryone else rather than fixing my own and moving on. Every number here was read on 29\nSeptember 2026 from endpoints that need no key, and the\n[model list I checked](https://freemodel.online/v1/models) is one of them.*", "url": "https://wpnews.pro/news/your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly", "canonical_source": "https://dev.to/yummy342/your-gateways-public-model-list-is-a-contract-most-of-them-break-it-quietly-29kc", "published_at": "2026-09-29 08:09:31+00:00", "updated_at": "2026-09-29 08:16:48.982580+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["OpenRouter", "OpenAI", "Anthropic", "openai-python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly", "markdown": "https://wpnews.pro/news/your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly.md", "text": "https://wpnews.pro/news/your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly.txt", "jsonld": "https://wpnews.pro/news/your-gateway-s-public-model-list-is-a-contract-most-of-them-break-it-quietly.jsonld"}}