{"slug": "stop-building-agents-that-cant-change-models", "title": "Stop building agents that can’t change models", "summary": "Enterprises should treat large language models as replaceable compute behind stable interfaces rather than as the foundation of their agent architectures, because model leadership and pricing shift faster than enterprise software contracts renew, according to Menlo Ventures data cited in a new analysis. Menlo Ventures found OpenAI's share of enterprise LLM API usage fell from 50% in 2023 to 27% by the end of 2025, while Anthropic rose from 12% to 40% and Google from 7% to 21%, and Stanford's 2025 AI Index put GPT-3.5-level MMLU performance at $20 per million tokens in November 2022 versus $0.07 by October 2024. The analysis points to concrete churn — Anthropic retiring Claude Sonnet 4 and Claude Opus 4 from its API on June 15, 2026 after April 14 notice, Claude Opus 4.1 following on August 5, and OpenAI shutting down its Assistants API in favor of the Responses API — and proposes a provider-agnostic ModelClient contract in Python that adapters translate into each vendor's tool-calling format.", "body_md": "No one can predict which AI provider will win the next benchmark cycle. The smarter approach is to build an architecture that can adapt as models change. One with a stable contract between agents and models, tools and state that exist outside any single provider, and evaluations that make each model change a measurable decision. The goal isn’t to avoid choosing a preferred model. It’s to ensure that when the next model emerges, changing that choice doesn’t require rebuilding the system around it.\n\nFor example, on June 15, 2026, Anthropic retired Claude Sonnet 4 and Claude Opus 4 from its API, after giving developers notice on April 14. Any application still calling those model IDs started receiving errors. Claude Opus 4.1 followed on August 5. Later that month, OpenAI shut down its entire Assistants API and pointed developers to its Responses API. None of this was a surprise. Each provider published dates and migration guides well ahead of time.\n\nThe problem sits on the builder’s side. Many teams designed agents where the model is the foundation and not a component, so when the foundation moves, the whole building moves with it. The model market is moving too fast for that design, and enterprises should treat models as replaceable compute behind stable interfaces. By doing so, they can change providers for cost, capability, privacy or performance without rebuilding their agents.\n\nAccording to [Menlo Ventures](https://menlovc.com/perspective/2025-the-state-of-generative-ai-in-the-enterprise/), OpenAI held 50% of enterprise large language model (LLM) API usage in 2023. By the end of 2025, that share had fallen to 27%. Anthropic climbed from 12% to 40%, and Google went from 7% to 21%. The data clearly indicates that leadership in this market changes hands faster than most enterprise software contracts renew.\n\nPrices move faster still, as [Stanford’s 2025 AI Index](https://hai.stanford.edu/ai-index/2025-ai-index-report) found that GPT-3.5-level performance on the MMLU benchmark cost $20 per million tokens in November 2022 and $0.07 by October 2024. Epoch AI found that the price of reaching a fixed performance level fell between 9x and 900x per year, depending on the task.\n\nThe best model for a task today will not be the best model for that task in 18 months, and architectures that assume otherwise are betting against the most consistent trend in the industry.\n\n[Menlo’s mid-2025 survey](https://menlovc.com/perspective/2025-mid-year-llm-market-update/) found that only 11% of teams had changed model providers in the prior year, while 66% upgraded within their existing vendor. What might be concluded here is that portability does not matter. However, many read it the other way. A low switching rate in a market with this much movement indicates switching is harder than it looks. And upgrading within one vendor is also a migration, because a new version changes output style, tool-calling behavior, latency and cost. An architecture that makes a cross-vendor switch cheap makes a same-vendor upgrade inexpensive too.\n\nIn this case, lock-in builds up through small, reasonable shortcuts such as:\n\nThe fix is an old systems idea. Put a stable contract between parts that change often and parts that should not. For example, here is a minimal sketch in Python:\n\n``` python\nfrom dataclasses import dataclass, field\nfrom typing import Protocol\n\n@dataclass\nclass ToolSpec:\n    name: str\n    description: str\n    json_schema: dict\n\n@dataclass\nclass ModelRequest:\n    system: str\n    messages: list[dict]\n    tools: list[ToolSpec] = field(default_factory=list)\n    task: str = \"default\"\n\n@dataclass\nclass ModelResponse:\n    text: str\n    tool_calls: list[dict]\n    input_tokens: int\n    output_tokens: int\n    model_id: str\n\nclass ModelClient(Protocol):\n    def complete(self, request: ModelRequest) -> ModelResponse: ...\n```\n\nTools use plain JSON Schema, and each provider adapter translates that into its own format. The “Task” label lets a router pick a model per task and lets the evaluation suite group results. The response records token counts and the “model_id that served the call, which makes cost tracking and regression analysis possible after a switch.\n\nThe agent never imports a provider SDK. A configuration file maps each task to an adapter. Changing providers becomes a configuration change followed by an evaluation run. Open-source gateways and OpenAI-compatible endpoints already handle much of this. The principle matters more than the library.\n\nThe Model Context Protocol (MCP) gives agents a standard way to call tools and data sources. Anthropic contributed it to the Linux Foundation’s Agentic AI Foundation (AAIF) in December 2025. Tools behind MCP servers work with any compliant client. The Agent2Agent (A2A) protocol, which reached version 1.0 in March 2026 and joined the AAIF in August, lets agents from different vendors hand off work.\n\nState belongs in systems you run, so keep conversation history, memory and source documents in your own databases and open formats. Embeddings are specific to the model that produced them, so switching embedding models means regenerating vectors. That job is routine when the source data is organized, and painful when the only copy lives in a provider’s hosted store.\n\nSwapping an adapter takes an afternoon. Knowing whether the new model is good enough takes a test suite of real task examples, scored with checks your team controls, with cost and latency recorded for every run. With that in place, a retirement notice becomes routine. In fact, Anthropic commits to at least 60 days of notice for publicly released models. However, sixty days is plenty for a team with an evaluation suite, and very little for one without.\n\nThen there is the portability cost. Providers ship features that exist nowhere else, such as specific prompt caching behavior or reasoning controls. A strict lowest-common-denominator interface throws those away. Let adapters declare optional capabilities, and let the router prefer them when useful. The agent still works without them, but it just costs more or runs slower, and that tradeoff stays visible.\n\nKeeping a base prompt per task with small per-model overrides in configuration helps keep things in check. When an override grows larger than the base prompt, treat it as a warning. The adapter layer also needs an owner, but that cost remains small next to a forced rewrite.\n\n| **Trigger** | **What the architecture must support** | \n| Cost | Per-task routing and cost tracking | \n| Capability | An evaluation suite that surfaces gaps | \n| Privacy and residency | Adapters for self-hosted and regional endpoints | \n| Performance | Latency metrics per model and task | \n\nA team that can move a sensitive workload to a self-hosted open-weight model, while keeping a hosted frontier model for everything else, has options that a single-provider team lacks.\n\nNobody can predict which provider wins the next benchmark cycle, and a sound architecture does not need that prediction. It needs a stable contract between agent and model, tools and state outside any single provider, and evaluations that turn every model change into a measurable decision. Teams that build this way still pick favorites. They just choose on the merits every quarter, instead of living with a choice that costs too much to change.", "url": "https://wpnews.pro/news/stop-building-agents-that-cant-change-models", "canonical_source": "https://www.infoworld.com/article/4239696/stop-building-agents-that-cant-change-models.html", "published_at": "2026-10-26 09:00:00+00:00", "updated_at": "2026-10-26 09:11:13.762326+00:00", "lang": "en", "topics": ["ai-agents", "large-language-models", "ai-infrastructure", "developer-tools", "ai-products"], "entities": ["Anthropic", "OpenAI", "Google", "Menlo Ventures", "Stanford HAI", "Epoch AI", "Claude Sonnet 4", "Claude Opus 4"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/stop-building-agents-that-cant-change-models", "markdown": "https://wpnews.pro/news/stop-building-agents-that-cant-change-models.md", "text": "https://wpnews.pro/news/stop-building-agents-that-cant-change-models.txt", "jsonld": "https://wpnews.pro/news/stop-building-agents-that-cant-change-models.jsonld"}}