Charles Holive, chief AI officer at BNP Paribas CIB, told Business Insider that he prioritizes dollar-value and productivity outcomes over raw token consumption. Holive is quoted saying, "We try to go away from vanity metrics - billions of tokens per day," and, "We try to make sure that what we track is an outcome, not a vanity metric," according to Business Insider. He also said he asks teams, "What did you do that you didn't do before? How much faster did you do it?" Business Insider places the remarks alongside wider industry pushback on usage-driven incentives, noting that Amazon recently shut down an internal program after employees reportedly gamed it to climb the rankings, that Uber operating chief Andrew Macdonald has questioned whether rising AI costs are producing useful products, and that GitHub moved Copilot to usage-based pricing, all per Business Insider.
What happened
Charles Holive, chief AI officer at BNP Paribas CIB, told Business Insider that he judges AI success by business outcomes rather than raw token consumption. Per Business Insider, Holive said, "We try to go away from vanity metrics - billions of tokens per day," and, "We try to make sure that what we track is an outcome, not a vanity metric." He told the outlet he presses teams with questions such as, "What did you do that you didn't do before? How much faster did you do it?" Business Insider reports the bank still tracks token usage but does not treat it as the primary measure of a project's value.
The broader pushback
Holive's comments land amid wider corporate skepticism about usage-driven AI incentives. Business Insider notes that Amazon recently wound down an internal program after employees reportedly gamed it to climb usage rankings, that Uber operating chief Andrew Macdonald has questioned whether rising AI spend is yielding useful products, and that GitHub shifted Copilot to usage-based pricing. The independently documented Copilot change took effect on June 1, underscoring how central token economics have become to enterprise AI budgets.
Why it matters for practitioners
Editorial analysis: For engineering and data teams, the shift moves attention from raw consumption telemetry toward outcome instrumentation, measuring time saved, task throughput, error reduction, and revenue impact. Tying individual model calls to concrete business events, then validating lift through A/B testing, is what lets teams attribute value to LLM features rather than inferring it from token volume.
What to watch
Editorial analysis: Expect more enterprises to publish outcome-linked AI KPIs, more product teams to adopt experiment frameworks that report business lift, and more vendors to refine pricing so that heavy token use is no longer mistaken for productivity. Dashboards that expose per-feature ROI will make it easier to compare consumption against the business value it actually generates.
Scoring Rationale #
One bank's chief AI officer reframing success around outcomes rather than token volume is a useful, on-trend enterprise-AI signal, but it is single-source executive commentary rather than a major event. Its relevance is amplified by the active 2026 tokenmaxxing and AI-ROI debate, which independently verifiable moves like GitHub's June 1 Copilot pricing shift underscore. Scored as a solid, niche-but-relevant practitioner data point.
Practice interview problems based on real data
1,500+ SQL & Python problems across 15 industry datasets — the exact type of data you work with.