[ Thought Leaders
](https://www.unite.ai/series/thought-leaders/)
[Add Unite.AI to your preferred sources on Google](https://www.google.com/preferences/source?q=unite.ai)
Until very recently, boardroom conversations around AI centered around the models themselves. Is their output accurate? How clean was the training data? How can we reduce bias? Are hallucinations under control? Those questions still matter, of course, but output is no longer the only thing boards and shareholders are paying attention to. Instead, they’re becoming interested in something that’s much harder to measure – whether AI systems are actually working *with *people as effectively as they could be in day-to-day interactions.
That’s partly why IBM recently found itself facing a shareholder proposal demanding greater transparency around how it manages AI bias and governance. The proposal focused specifically on bias mitigation, but the broader takeaway was that as AI becomes more deeply embedded in decision-making processes, it’s being judged based on its operational accountability and measurable business outcomes rather than its basic level of capability.
What I’m seeing in conversations with enterprises is that very few people are still debating whether the models themselves are capable. Most organizations already know these systems can generate outputs, automate workflows, and surface insights at extraordinary speed. The harder question is whether any of that is consistently improving outcomes inside the business. Are employees actually using these systems in meaningful ways? Do they trust the recommendations they’re receiving? Are decisions becoming clearer, faster, or more consistent? Or are organizations simply generating more content, more analysis, and more noise without changing how people operate? That gap between AI capability and human adoption is what I think of as the “last mile” of enterprise AI. It’s where ROI, trust, accountability, and operational consistency converge, and it’s where many deployments begin to stall.
Capability isn’t the bottleneck – human context is #
The IBM shareholder proposal itself centered on bias, a familiar and growing concern in the industry. The resolution called on IBM to produce additional reporting on the methods it uses to identify and mitigate bias across its AI models, including the potential risks associated with those mitigation efforts. IBM pushed back, arguing that much of this information was already publicly available through model cards, governance documentation, transparency reporting, and submissions to Stanford’s Foundation Model Transparency Index. Most observers acknowledged IBM has done more than many vendors to operationalize responsible AI practices, while also recognizing that governance and bias remain unresolved industry-wide challenges.
What’s interesting to me isn’t whether IBM’s response was right or wrong. It’s what the entire situation says about where enterprise AI conversations are heading. Boards have largely moved into a new phase of AI evaluation focused on decision quality, operational impact, employee trust, and long-term adoption inside the business.
This is where a neuroscience lens leads me to a different conclusion than much of the current enterprise AI conversation. Humans are not rational input-output machines. We approach decisions with finite attention, fluctuating confidence, cognitive overload, and constantly shifting intent. When interactions feel slow, when canned responses miss what we actually meant, or when people have to fight the system to get to something useful, engagement collapses long before adoption metrics catch up. Hallucination risk, inconsistency, and bias simply compound the problem, creating friction inside workflows that were originally intended to improve productivity and decision-making. This “last mile” is what most organizations still underestimate: AI value isn’t created when a model generates an answer, but when it delivers the right answer at the right time and humans can trust it enough to act on what it produces.
Why are we ignoring the real ‘human in the loop’ – the user? #
We’ve been hearing a lot recently about “human-in-the-loop” and the importance of having a human present to sign-off on AI-driven output. But what if the most important human – the user – isn’t being properly understood? Human-to-human interaction is complex and multi-faceted, but the reason direct communication works so well is because no nuance is lost. If someone hesitates, loses confidence, becomes frustrated or confused, or their intent shifts, it carries through during an interaction and the conversation shifts accordingly. Most AI systems still struggle to recognize those signals because they’re more or less “on rails” in that all they’re responding to is the last prompt from the user.
In my view, this is becoming one of the defining limitations of current enterprise AI deployments. We’ve built systems that are exceptionally good at generating information while remaining limited in their ability to recognize whether that information is helping someone make better decisions.
An organization can deploy enterprise AI tools for thousands of employees and still fail to achieve meaningful alignment in how those systems are actually being used. Frameworks such as the NIST AI Risk Management Framework, ISO/IEC 42001, and the EU AI Act are pushing organizations toward greater governance maturity and operational accountability. But governance on paper only goes so far if employee engagement with AI remains fragmented. If different teams trust and use AI in completely different ways, measuring ROI consistently becomes almost impossible, and that’s a problem no amount of prompting can fix.
The cost of AI overload #
We are now entering a phase where AI is becoming completely ubiquitous inside of AI workflows. There are copilots embedded into productivity suites, AI-generated meeting summaries arriving after every call, dashboards surfacing predictive insights in real time, recommendation engines feeding employees constant suggestions, and automated analysis appearing faster than most teams can realistically process it. On paper, this looks like progress – businesses are producing more information than ever before. But cognitive science has been clear about this for decades: more information does not automatically produce better decisions.
People can only absorb so much information before overload starts to set in. Too many recommendations create hesitation. Too many alerts become background noise. Too many AI-generated options can leave employees second-guessing rather than acting confidently. What’s interesting is that AI systems are becoming exceptionally good at producing information while still remaining relatively poor at understanding how humans receive, interpret, and respond to that information in context. Boards ultimately don’t care how many summaries, recommendations, or generated insights a system can produce. They care whether decisions are becoming clearer and more reliable across the business.
AI has entered its accountability phase #
The IBM shareholder proposal may have come from concerns about bias, but I think it reflects something far broader that’s happening right across enterprise AI. The novelty of AI has passed, and enterprises looking for a solid return on their AI investment can no longer look to model capability alone. The technology has proven itself, but now it needs a measure of accountability. How consistently are employees using these systems? Are decisions improving? Are trust and engagement increasing over time, or gradually eroding? As AI becomes more embedded in everyday workflows, the next generation of systems may need to become far more aware of human behavior, attention, confidence, and intent if businesses want sustained adoption and measurable ROI. Until then, the last mile will likely remain one of the biggest challenges in enterprise AI adoption.