AI Infrastructure Spending: The Blind Race to Scale

Enterprises are rapidly investing in AI infrastructure, but a new report reveals that most lack the tools to measure its economic impact, with 83% reporting GPU utilization at 50% or less and fewer than half tracking compute costs. The race to scale is driving a re-platforming wave, as 64% of enterprises plan to switch or add an infrastructure provider within the next year, prioritizing integration and total cost of ownership over headline prices.

AI Infrastructure Spending: The Blind Race to Scale Enterprises are racing to invest in AI infrastructure, but most lack the tools to measure its economic impact. A re-platforming wave is imminent. AI infrastructure spending is on the rise, with 107 enterprises leading the charge. Yet, most are investing faster than they can track the economics of their decisions. Companies are flocking to familiar hyperscalers and model-provider APIs, but the next wave of expenditure is shifting toward specialized compute /glossary/compute that remains largely unused today. The Compute Gap: Investment Outpaces Insight The data shows a stark reality: while 83% of enterprises report GPU /glossary/gpu utilization at 50% or less, fewer than half can rigorously account for what their AI compute costs. Only 21% of enterprises run AI in production at scale, yet there's a mad dash to evaluate AI-specialized clouds, with 45% planning assessments in the next year. Essentially, businesses are investing in infrastructure faster than they can comprehend its financial implications. As enterprises rapidly spend on AI infrastructure, many lack the visibility to gauge unit economics, which is important for informed decision-making. This compute gap signifies a disconnect between ambitious investment and the reality of underutilized resources. Changing Providers: The Rush to Replatform The competitive landscape shifted this quarter, with 64% of enterprises planning to switch or add an infrastructure provider within the next year. An impressive 38% intend to make changes within just three months. Why the rush? Enterprises prioritize integration and total cost of ownership over headline prices, with only 8% considering cost per token /glossary/token as a critical factor. However, the allure of AI-specialized clouds, which have historically seen marginal use, is gaining traction. It's a re-platforming moment. But are these organizations merely reshuffling the deck chairs without a clear view of the iceberg that lies ahead? The Blind Spot: Measuring Costs and Returns Here's how the numbers stack up. Less than half of enterprises track their compute costs and returns thoroughly. With total cost of ownership being a primary concern, the gap in measurement poses significant risks. How can organizations optimize their AI investments if they can't grasp the financial picture? This oversight extends to the next constraint on the horizon: the shift from GPU compute to memory in large-scale inference /glossary/inference , a shift that many enterprises haven't yet acknowledged. This blind spot may widen the current compute gap, especially as companies pile on more infrastructure without understanding its costs. The market map tells the story. Enterprises are betting big on AI infrastructure, but the lack of economic visibility could lead to inefficient spending. Will organizations address these gaps before the re-platforming wave hits, or will they remain blind to the true costs of their AI ambitions? Get AI news in your inbox Daily digest of what matters in AI.