{"slug": "determining-the-roi-of-ai-requires-data-that-most-companies-lack", "title": "Determining the ROI of AI requires data that most companies lack", "summary": "Most organizations cannot determine the ROI of their AI initiatives because they lack the necessary data, according to an analysis by InfoWorld. Unlike cloud computing, which requires only cost and business data to calculate ROI, AI requires a third data source—telemetry—that most companies do not capture. Without telemetry data that maps costs to specific customers, features, or outcomes, enterprises cannot attribute AI expenses to business value, leaving CFOs unable to answer which AI initiatives are profitable.", "body_md": "Leadership wants to scale AI. Budgets are tripling. Adoption is up.\n\nThen the CFO asks the question every board now asks: which of these initiatives is actually profitable?\n\nMost organizations cannot answer that question, not because they lack visibility into cost, but because the cost data they have was never designed to produce that answer.\n\nApplying lessons learned from [managing cloud spend](https://www.infoworld.com/article/4147766/cloud-at-20-cost-complexity-and-control.html) won’t be a fix for the AI and ROI quandary. True, cloud taught a generation of CFOs that billing without business context is noise. So to get [cloud ROI](https://www.infoworld.com/article/4061122/cloud-computing-has-an-roi-problem.html), they stitched two data sources together: cost data plus business data. AWS reveals which account, which region, which tag, which resource. Merge in customer and product mappings on top and the ROI of the cloud spend comes into focus.\n\nBut AI is harder. It requires three data sources: cost, business, and telemetry—the automatic collection of data from disparate sources that helps to clarify the whole picture of what happened and why. An executive or engineering lead can have AI invoices and customer revenue. But they have no way to connect them to business value. The token count on the OpenAI invoice does not specify which customer triggered which call, which feature it served, or whether the prompt produced a business outcome. That data does not exist in the provider’s billing.\n\nThe situation is not likely to change anytime soon because AI providers are not in the business of attributing an enterprise’s costs to that enterprise’s customers. Instead, AI providers are in the business of selling tokens. The granularity they expose is the granularity their billing systems require, not the granularity a CFO requires.\n\nNot convinced? Compare what AWS gives you to what an AI provider gives you.\n\nAWS billing exposes resource IDs, account hierarchies, region, SKU, tag metadata, usage by the minute. Every dollar can be attributed to a workload, a team, a customer segment if it was tagged correctly. The data is rich enough that mature FinOps teams built unit economics on top of it years ago.\n\nAn AI provider invoice gives you tokens consumed by model, with optional grouping by API key. That is the resolution. No request-level attribution. No customer ID. No feature mapping. No prompt outcome. No retry identification. Multi-step agent workflows collapse into a token count. Imagine a large bank receives a multi-million dollar AI invoice each month. But it has no visibility into what parts of the business were responsible for what parts of the cost so cannot allocate them.\n\nIf an enterprise wants to know what AI cost drove which customer or feature, it has to capture that data itself, inside an application, before the call leaves it.\n\nBuilding AI ROI measurement requires three data sources, stitched together in a single model.\n\nStitched together, the three sources produce the unit economics every AI investment decision now requires: cost per customer interaction, margin per feature, profitability per agent workflow, ROI per model choice. None of these can be calculated from billing data alone. None can be calculated from telemetry alone. They require all three sources, modeled together in a way that maps cost to outcome.\n\nSingle-call inference is the easy case. One request, one cost, one customer, one outcome.\n\nAgentic workflows are different. An agent decomposes a task into multiple steps. Each step calls a model. Some steps fall back to a different model when the first fails. Some steps retry on a poor result. Some steps invoke external tools that themselves cost money. A single user request can produce dozens of inference calls across multiple providers, with the cost compounding in ways the provider invoice cannot disaggregate.\n\nIf telemetry does not capture agent-step granularity, no one will know which steps are profitable. Aggregate costs will show up three weeks later in the invoice. By then, the workflow has been running at scale, customers are onboarded, and unprofitable paths have been retried thousands of times.\n\nWhen agents make the calls, the volume of cost-generating events without business context attached grows by an order of magnitude. The window for instrumenting this before it becomes unmanageable is closing.\n\nOnce the three sources are stitched together, the AI investment conversation changes.\n\nFive different ways to build the same AI capability stop looking equivalent. They converge on adoption metrics and diverge by 10x on cost. The team picks the approach that delivers a similar business outcome at one-fifth the cost, because the team can finally see the difference. Product teams design features with margin awareness from the architecture phase, not from the post-launch budget review. Engineering teams choose model architectures with cost-per-outcome data alongside latency and quality. Leadership evaluates AI initiatives the way they evaluate any other capital allocation: on unit economics, not on the engagement chart. Aggregated invoices track the cost per customer interaction. Engagement metrics reveal margin per feature. Gut-instinct model selection is checked against real cost-per-outcome model selection results.\n\nWithin seconds, everyone can see which AI features are profitable, which should scale, and which should be killed. This is the insight everyone is looking for and companies that achieve it will optimize the benefits of AI.\n\nAI costs are compounding now. The board is not waiting 18 months for an internal project to reach production.\n\nThe temptation to build it anyway has never been sharper. AI coding tools have changed what a small engineering team can ship in a quarter. The instrumentation layer looks tractable. The cost normalization looks like a weekend project. The semantic model feels like something a senior engineer could draft over a sprint.\n\nIt is a trap. Three reasons.\n\nVolume is the first. A production AI footprint generates millions of telemetry events per hour, and that volume scales with agentic adoption. Real-time ingestion, correlation, and attribution at that scale is not the same problem as [vibe coding](https://www.infoworld.com/article/4078884/what-is-vibe-coding-ai-writes-the-code-so-developers-can-think-big.html) a prototype in an afternoon. It is a permanent operational system that has to be right every minute of every day.\n\nThe vendor landscape is the second. Cost data arrives in delayed billing windows from providers with non-interoperable schemas. Schemas change without notice. New AI providers enter the landscape monthly, each with its own taxonomy and metering. The system is not built once. It is maintained against a moving target that moves faster than most internal release cycles.\n\nThe third is what the first two add up to: this is business-critical infrastructure. The CFO and the board are going to make capital allocation decisions on the data this system produces. When schema drift goes unnoticed for two weeks, when an agent telemetry stream stops correlating to a vendor that quietly changed its billing API, the cost of being wrong is not a sprint of cleanup. It is a quarter of misallocated capital.\n\nThe build-vs.-buy question for engineering leaders has changed. It’s not “can we build this?” The honest answer is yes. The real question is whether the marginal hour of your strongest engineers is best spent stitching cost data to telemetry to business outcomes, or building the AI products that produce the revenue the cost data is measuring.\n\nThe capability is reproducible in weeks. The choice is whether to spend the next 18 months building it, or the next 18 months acting on it.\n\n*—*\n\n*New Tech Forum*** provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all ****inquiries to *** doug_dineley@foundryco.com***.**", "url": "https://wpnews.pro/news/determining-the-roi-of-ai-requires-data-that-most-companies-lack", "canonical_source": "https://www.infoworld.com/article/4200283/determining-the-roi-of-ai-requires-data-that-most-companies-lack.html", "published_at": "2026-07-23 09:00:00+00:00", "updated_at": "2026-07-23 09:22:15.399085+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-policy"], "entities": ["InfoWorld", "AWS", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/determining-the-roi-of-ai-requires-data-that-most-companies-lack", "markdown": "https://wpnews.pro/news/determining-the-roi-of-ai-requires-data-that-most-companies-lack.md", "text": "https://wpnews.pro/news/determining-the-roi-of-ai-requires-data-that-most-companies-lack.txt", "jsonld": "https://wpnews.pro/news/determining-the-roi-of-ai-requires-data-that-most-companies-lack.jsonld"}}