Grounding, not models, will define your AI advantage

Enterprises are making a costly mistake by prioritizing custom large language models over grounding infrastructure, according to a product manager with years of AI experience. As model costs collapse by over 90% by 2030, competitive advantage shifts to proprietary data pipelines and retrieval-augmented generation (RAG) systems that compound in value. Companies that invest in grounding, not models, will build durable AI moats.

Over the past two years, working inside the enterprise AI infrastructure world, tracking where the industry is heading, I have noticed the same question surface repeatedly: should we build our own large language model? I understand the instinct. The model feels like the thing, the engine, the brain, the asset worth owning. But after significant years as a product manager in the AI world in both customer experience and grounding infrastructure I concluded that it tends to unsettle the room: the model is the least durable part of your AI strategy. I say this not to be provocative, but because over the last few years we have seen organizations pour their scarcest resources, executive attention, engineering talent, capital, into the one layer of the stack that is commoditizing fastest. Meanwhile, the layer that determines whether their AI is trustworthy, accurate and defensible gets treated as plumbing. That inversion is, in my experience, the single most expensive mistake enterprises are making with AI right now. Let us consider economics. Gartner projects that by 2030, performing inference on a trillion-parameter model will cost providers more than 90% less https://www.gartner.com/en/newsroom/press-releases/2026-03-25-gartner-predicts-that-by-2030-performing-inference-on-an-llm-with-1-trillion-parameters-will-cost-genai-providers-over-90-percent-less-than-in-2025 than it did in 2025, with models becoming up to 100 times more cost-efficient than the earliest versions of comparable size. When the cost of the underlying capability collapses by that magnitude, it stops being a differentiator. Anything that gets that cheap, that fast, is not where competitive advantage lives. Models that feel innovative are routinely surpassed by something cheaper and better within months. If your advantage is tied to a specific model, it will evaporate the moment the frontier moves, which it always does. But if an enterprise instead invests in how reliably it can feed any model its proprietary context, that investment holds. That part travels from one model generation to the next. When a better model arrives, the organization can simply connect it and immediately capture the upside, because the hard and durable work was already done one layer down. I wish more leaders could observe this pattern before they commit. The model layer is improving so quickly that any advantage you build into it has a short half-life. The grounding layer behaves in the opposite way: every improvement you make to your data quality, your retrieval logic and your governance compounds, and it carries forward regardless of which model sits on top. This is why the build-your-own LLM debate so often misses the mark. Training or even meaningfully fine-tuning a foundation model is enormously expensive, and the moment you finish, the open and commercial frontier has usually moved past you. So, technically you spent a fortune to own a depreciating asset. The capability that you should focus on is an AI that knows your business, was never going to come from the weights of the model anyway. It comes from what you put in front of it. Grounding is the discipline of connecting a general-purpose model to your enterprises’ current and authoritative information, most commonly through retrieval-augmented generation, or RAG. Rather than hoping the model memorized something useful during training, you retrieve the relevant facts from your own systems in real time of the query and give the model the context it needs to answer correctly. Here is the part that matters for anyone thinking about competitive advantage: your competitors can rent the exact same model you use. What they cannot rent is your data, your institutional knowledge, your processes and the quality of the pipeline that surfaces all of it accurately at the right moment. That pipeline is genuinely proprietary, genuinely hard to replicate and it compounds in value over time. That is the textbook definition of a moat, and it has almost nothing to do with which model you chose. The industry is starting to recognize this. Gartner predicts that by 2027, organizations will use small, task-specific models at least three times more than general-purpose LLMs https://www.gartner.com/en/newsroom/press-releases/2025-04-09-gartner-predicts-by-2027-organizations-will-use-small-task-specific-ai-models-three-times-more-than-general-purpose-large-language-models , precisely because accuracy in real business workflows depends on domain context rather than raw model scale. But a smaller model holds less in its parameters by design, which means it leans even harder on retrieval to supply current, authoritative context in real time. The model gets smaller and more swappable. The grounding becomes the part that carries the weight. In that same analysis, Gartner makes the same point from the data side: what sets enterprises apart is how well they prepare, check, version and manage their own data. Read that again: the differentiator is the data discipline, not the model. This matches what I have observed directly. Getting hold of an excellent model was never the hard part, and it was rarely where things broke. The failures I have seen came from not connecting the model efficiently to the right data sources or orchestrating retrieval well. The patterns repeat: missing data produces incomplete summaries, truncated documents leave answers without key details, and noisy context yields irrelevant or confusing responses. When grounding is absent, answers become inconsistent from one client to the next; when retrieval comes back empty, the model fills the gap with something hallucinated or useless. Stale data produces confidently outdated answers, retrieval gaps surface as generic non-answers, and poor-quality data drags down both speed and output. None of these are model problems. They are grounding problems. And when a system hands an executive an answer that is wrong, no one in the boardroom cares how sophisticated the model was. They care that it was wrong, and the fix always lives in the grounding layer. One example has stayed with me. In a real enterprise scenario, an AI assistant returned inconsistent answers to the same query across different environments whenever grounding was unavailable, and some of those answers contradicted each other outright. The cause was straightforward in hindsight. With no grounding, the system fell back on its own internal knowledge instead of a shared, grounded source of truth, so its responses drifted with each configuration and context. The damage was not just technical. Users stopped trusting an assistant that could not give them the same answer to the same question twice. That is the actual cost of weak grounding, and it is why consistency and reliability in production depend far more on the data layer than on the model sitting above it. No model upgrade would have fixed that. If you accept that grounding is where advantage accrues, a few priorities shift in ways that should change how you allocate budget and attention. First, treat your organization’s data foundation as a first-class AI investment, not a prerequisite you rush through. The unglamorous work, cleaning, structuring, governing and versioning your knowledge, is the work that determines AI quality. I would rather inherit a mediocre model with an excellent retrieval pipeline than the reverse, every single time. Second, build for model portability from day one. Assume the model you use today will be replaced within a year because it certainly will. If swapping it out is painful, you have coupled your architecture to the wrong layer. Your grounding infrastructure, your evaluation framework and your data contracts should be the stable core; the model should be a component you can swap with minimal disruption. Third, invest in observability and evaluation for retrieval, not just for the model. The emerging discipline here matters: Gartner expects LLM observability investments to reach 50% of GenAI deployments by 2028 https://www.gartner.com/en/newsroom/press-releases/2026-03-30-gartner-predicts-by-2028-explainable-ai-will-drive-llm-observability-investments-to-50-percent-for-secure-genai-deployment , up from 15% today, as trust requirements outpace the technology itself. Knowing why your system retrieved a particular piece of context, and whether that context was correct, is what makes an AI output defensible and auditable. For any organization operating under real regulatory or reputational scrutiny, that is not optional. None of this means the model is irrelevant. You still need a capable one and choosing well matters. But choosing a model is now a procurement decision with several excellent options, not a source of lasting differentiation. The lasting differentiation is everything you wrap around it. I think the organizations that internalize this will look, in a few years, meaningfully ahead of the ones still debating whether to train their own model. Not because they made a bolder bet, but because they made a more durable one. They understood that in a world where everyone has access to the same extraordinary models, the advantage belongs to whoever grounds those models best in the reality of their own business. The model is rented. The grounding is owned. Build accordingly. This article is published as part of the Foundry Expert Contributor Network. Want to join? https://www.cio.com/expert-contributor-network/