A survey of 700 engineering leaders and practitioners from organizations with more than 1,000 employees published this week suggests 26% of all spending on artificial intelligence (AI) is being wasted.
Conducted by Harness, a provider of a DevOps platform, the survey finds that organizations spending $1 million per month on AI are wasting approximately $260,000 a month with no measurable return.
Nearly three quarters of respondents also admit their organizations have experienced an unexpected AI cost spike or bill in the past year, with a third of organizations (33%) having been caught off guard more than once. A total of 80% of respondents work for organizations that require a full day or longer to trace the source of an AI cost spike, with 32% needing a full week.
More troubling still, more than half of respondents (52%) said there is no clear cost owner for AI within their organization, with responsibility split across engineering, FinOps, finance, and IT. Nearly three quarters (73%) have AI cost policies in place, but only 47% actively enforce them.
Only 26% have a robust method for measuring the business value of their AI spend, while more than half (56%) admit anticipating AI spend is based on guesswork, not data. On average, less than half (45%) say they understand the cost of the AI features they build, with well more than half (57%) of engineers saying their organization actively encourages “tokenmaxxing” without any sense of the business value being derived. Only 21% describe their current practices for tracking AI spending as fully mature.
Collectively, the survey results make it clear there is a compelling need to start applying best FinOps practices to AI, says Patrick Brogan, director of the FinOps Advisory practice for Harness. In much the same way organizations failed to control cloud spending, the same issues are now playing out as organizations race to build and deploy AI applications, he adds. “There’s a gap in governance,” says Brogan.
The challenge is that the amount of money being wasted on AI is substantially higher, notes Brogan. The issue is becoming more problematic as organizations start to better understand the total cost of building and deploying AI applications, he adds. The simple truth is that organizations need to be more judicious about which AI projects they can fund, says Brogan. The more budget dollars that wind up being wasted, the fewer AI applications will actually be deployed, he notes.
At the same time, organizations will also need to become savvier about rightsizing AI applications and models, says Brogan. Not every application requires access to the latest, most expensive AI model, and in many cases, the organization may find an AI application is more than adequately served by an open source model, he adds.
Each organization will, naturally, need to decide for themselves what the best path forward is, but if the choice comes down to reducing personnel to better afford AI model access, that organization might be better served by revisiting how their existing AI budget dollars are actually being spent before blindly throwing more money into the proverbial AI money pit.