{"slug": "dont-let-ai-negotiate-with-reality", "title": "Don’t let AI negotiate with reality", "summary": "Accenture's Change Reinvented research found that 95% of organizations underwent at least two transformations in three years, while only 30% of C-suite leaders expressed confidence in their change capabilities, and a McKinsey survey of over 1,200 executives found 40% expect significant business model changes within three years. The author argues that AI should help understand reality but not negotiate with it, citing OpenAI's rollback of a GPT-4o update in 2025 due to overly flattering and disingenuous responses, and calls for a separation of powers in AI-assisted decision-making.", "body_md": "We are all transforming now.\n\nSome companies have formally named transformation programs. Others are being transformed by a new regulation, an AI mandate, a cyber event, a weather disruption, a change in customer behavior, a competitor’s move or an urgent demand to reduce costs.\n\nThe label is almost beside the point. The operating assumptions keep changing, and the company has to change with them.\n\n[Accenture’s Change Reinvented research](https://www.accenture.com/se-en/insights/consulting/change-reinvented) found that 95% of organizations had undergone at least two transformations in three years, while only 30% of C-suite leaders expressed confidence in their organizations’ change capabilities.\n\nMore recently, a [McKinsey Global Survey of more than 1,200 executives and managers](https://www.mckinsey.com/capabilities/strategy-and-corporate-finance/our-insights/why-accelerated-resource-allocation-matters-in-the-age-of-ai) found that 40% expect their current business models to require significant change within three years simply to remain economically viable.\n\nTransformation is no longer an event that temporarily interrupts normal operations. It is becoming normal operations.\n\nThat changes the role AI is beginning to play. We are not using it only to draft emails, summarize documents or write code. We are increasingly asking it to interpret complex situations, identify options, recommend priorities and influence consequential business decisions.\n\nI believe that can be enormously valuable. I also believe it requires a boundary we have not defined clearly enough.\n\nAI should help us understand reality. It should not be allowed to negotiate with it.\n\nI have sat in versions of this meeting many times.\n\nThe company must reduce spending by 10%. A regulatory deadline cannot move. The CEO has declared AI a strategic priority. A customer initiative has already been promised to the market. Hiring is frozen. Several of the same architects, data specialists, cybersecurity professionals and change leaders are required by every program.\n\nEach commitment may be rational. Together, they may be impossible.\n\nSomeone asks the AI assistant to recommend a plan that protects all of them.\n\nThe answer arrives almost immediately. It proposes phased delivery, tighter governance, selective automation, resource sharing, increased collaboration and a revised sequence. It sounds balanced. It may even sound reassuring.\n\nBut did the answer prove that the commitments can coexist? Or did it produce the most plausible story that satisfies the request?\n\nThat distinction matters.\n\nIn 2025, [OpenAI rolled back an update to GPT-4o](https://openai.com/index/sycophancy-in-gpt-4o/) after concluding that the model had become overly flattering and agreeable. OpenAI described some of the responses as overly supportive but disingenuous and acknowledged that the model had been too influenced by short-term user feedback.\n\nThe point is not that AI cannot be trusted. The point is that an AI system can be highly intelligent, useful and well-intentioned while still being pulled toward the answer its user would prefer.\n\nThat is manageable when the stakes are wording or tone. It becomes dangerous when the question is whether the enterprise can afford, staff, sequence and deliver everything leadership wants.\n\nOrganizations need a separation of powers for AI-assisted decision-making.\n\nHuman judgment should establish intent. Leaders decide what matters, which outcomes deserve protection, what risks are acceptable and which tradeoffs the organization is willing to make. No mathematical model can decide what a company ought to value.\n\nGovernance should establish authority. It determines who may change a priority, move a date, redirect capital, accept more risk or relax a constraint.\n\n[PMI’s 2026 Closing the Change-Readiness Gap report](https://www.pmi.org/learning/thought-leadership/closing-the-change-readiness-gap) argues that enterprise agility depends on aligning intent, authority, structure and trust. Yet only 41% of executives surveyed believe their operating models support rapid allocation of capital and talent.\n\nThe ability to move resources quickly is important. So is the ability to see what that movement changes elsewhere.\n\nMathematics should establish feasibility. Once the assumptions, capacity, funding, dates, dependencies and constraints have been made explicit, the organization needs a protected calculation of whether its commitments can coexist.\n\nMath does not decide strategy. It does not make imperfect data perfect. It does not remove politics, judgment or uncertainty.\n\nIt does establish where judgment ends and wishful thinking begins.\n\nIf 12 initiatives need the same six specialists during the same quarter, the organization does not have a communication problem. It has a capacity collision.\n\nIf a budget reduction removes the resources needed to achieve the original business case, the economics have changed even if the presentation has not.\n\nIf two regulatory commitments depend on the same release window, confidence will not resolve the sequence.\n\nAI should establish understanding. It can question assumptions, find inconsistencies, identify patterns, propose alternatives and explain why an option succeeded or failed. It can help leaders ask better questions and explore complexity without waiting for days of manual analysis.\n\nBut the sequence matters.\n\nAI may recommend that an assumption change. It should not silently change that assumption to produce a more acceptable answer.\n\nCIOs are accustomed to protecting data. The next challenge is protecting the authority of different kinds of information.\n\nA recorded fact is not the same as an assumption. An approved risk tolerance is not the same as an executive preference. A mathematically calculated shortfall is not the same as an AI-generated interpretation. A proposed option is not a commitment.\n\nWhen all of these appear in one polished response, the distinctions can disappear.\n\n[EY’s analysis of the 2026 COSO framework](https://www.ey.com/en_us/cro-risk/coso-2026-and-the-shift-in-ai-governance) makes this problem tangible. EY argues that control for AI-enabled decisions must move upstream, preserving evidence of the inputs, model outputs, human review, exceptions and changes that shaped the judgment—not merely documenting approval after the decision has been made.\n\nThat is a useful way to think about a protected truth layer.\n\nThe AI should be free to interrogate the facts, challenge assumptions, recommend alternatives and explain consequences. But changes to a date, budget, dependency, constraint or risk tolerance should remain visible, attributable and governed.\n\nOtherwise, the enterprise may believe it is evaluating a new option when the AI has actually altered the question.\n\nSpeed makes the distinction more important, not less. [West Monroe’s 2026 Speed Wins research](https://www.westmonroe.com/press-releases/slow-decisions-are-costing-companies-millions) found that more than 1,200 leaders reported losing up to 5% of annual revenue because decisions and execution move too slowly.\n\nOrganizations do need to decide faster.\n\nBut accelerating the conversation without protecting its underlying truth can simply produce a bad decision sooner.\n\nBefore allowing AI to influence major transformation or portfolio decisions, CIOs should be able to answer four questions:\n\nIf the answer to any of these is no, the organization may have an intelligent conversational interface. It does not yet have a trustworthy decision capability.\n\nThis is not an argument for keeping AI out of the decision room. Continuous transformation may make AI indispensable. The volume of change, the number of moving parts and the speed of interaction across an enterprise are becoming too great for people to process unaided.\n\nBut AI cannot be the source of the facts, the interpreter of the facts, the judge of feasibility and the author of the recommendation without clear boundaries among those roles.\n\nHumans must retain responsibility for intent and judgment. Governance must make authority and changes explicit. Mathematics must test whether commitments can coexist. AI should make the resulting complexity easier to explore, understand and act upon.\n\nContinuous transformation requires a mechanism that can absorb a new condition, expose what it affects, test feasible responses and present credible options while the decision is still being made. AI can make that mechanism far more accessible.\n\nIt should not be allowed to make an impossible option sound possible.\n\nAs AI enters more executive decisions, the most important question may not be what the system can do.\n\nIt may be what the system is not allowed to negotiate.", "url": "https://wpnews.pro/news/dont-let-ai-negotiate-with-reality", "canonical_source": "https://www.cio.com/article/4208093/dont-let-ai-negotiate-with-reality.html", "published_at": "2026-08-12 11:00:00+00:00", "updated_at": "2026-08-12 11:20:05.494540+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-ethics", "ai-policy"], "entities": ["Accenture", "McKinsey", "OpenAI", "GPT-4o", "PMI"], "alternates": {"html": "https://wpnews.pro/news/dont-let-ai-negotiate-with-reality", "markdown": "https://wpnews.pro/news/dont-let-ai-negotiate-with-reality.md", "text": "https://wpnews.pro/news/dont-let-ai-negotiate-with-reality.txt", "jsonld": "https://wpnews.pro/news/dont-let-ai-negotiate-with-reality.jsonld"}}