Imagine working with someone who was completely unable to learn, to gain any experience whatsoever. There’s no need to picture someone completely stupid: just think of someone who joins the company, starts to deal with a lot of customers, experiences successes and failures, attends meetings, makes decisions . . . but one year later, has the same skills and capabilities as on day one. Would you be able to call that person experienced?
Surprisingly, that’s exactly what happens with much of the corporate AI that is being deployed and implemented these days. Obviously, I’m not saying that an AI system cannot change after deployment: we all know it can, we have experienced that even in our own interactions with chatbots. Systems can accumulate memory, retrieve previous interactions, be updated, fine-tuned or have their surrounding logic modified. However, remembering is not the same as learning. A system can accumulate information without becoming better at deciding what to do next. Using an LLM more does not, by itself, make the deployed model learn from the consequences of its actions.
Satya Nadella saw that clearly, and distinguished the replaceable general model from the enduring “company veteran” expertise that ought to accumulate around it.
How many streams of lessons does your company produce every day? From a disgruntled customer, to a discount that worked or didn’t work, a supplier that delivered late, a support operation that solved a problem or made it worse, a sales approach that failed in a segment but worked on a different one . . . All these things are not simply “data”: they are specific actions, followed by specific consequences.
How much of that experience can make tomorrow’s AI better than today’s? If we take into account what Microsoft has to say about this, the possible gains come when you learn what happened in production and you are able to feed those signals into prompts, routing, retrieval and other system decisions: everyday operations need to become lessons from which your system can learn.
Imagine a team of very disciplined salespeople that keep a notebook with their activity carefully annotated: after thousands of calls, the notebook will contain an awful lot of information, but the judgment of the salespeople will be pretty much the same. Have they really learned anything? Remembering that one specific customer rejected a 10% discount can be useful or practical, but learning that this particular customer segment responds better to faster implementations than to discounts is something qualitatively different.
There’s even a research field called continual learning, whose goal is to allow intelligent systems to learn incrementally from ever-changing streams of experience while retaining useful previous knowledge: basically, how to learn new things without forgetting old ones.
When different competitors can buy the same frontier models, access to these models does not constitute a sustainable competitive advantage anymore. What can become the essence of such an advantage is actually all the things you have been able to learn from your particular history: customers, decisions, campaigns, corrections, failures, exceptions, outcomes, etc.
Satya Nadella proposed an interesting “company veteran test” in this regard: imagine that tomorrow you replace a model with a different one, will the expertise accumulated inside your organization remain? The real opportunity lies in building a learning loop in which human and token capital can compound, rather than simply choosing the best model. Accumulated learning should persist outside the replaceable model rather than disappearing when you get a new one.
The idea is that scale should make the company smarter, not just bigger. Traditional software can process a million transactions instead of ten thousand. An intelligent system should go one step further: interaction number one million should leave it better prepared for interaction 1,000,001. Compounding intelligence is what you should get from your corporate system, instead of just databases, records or memory. Recommendation systems at companies such as YouTube, Spotify or Amazon have long used streams of user behavior and feedback to improve future recommendations (or at the very least, try to improve them), and they do so even without AI. Why should we accept corporate AI that doesn’t learn and improve from corporate experience?
Feedback loops can be bad as well as good. The company needs to decide what success actually means. Faster customer service calls are not necessarily better if the customers leave afterwards because they didn’t like the quality of the interaction. A higher conversion is not necessarily better if discounts are eroding our margins. More output is not better if, as a consequence, the quality gets worse. The British economist Charles Goodhart has a quote I use in all my courses: when a measure becomes a target, it ceases to be a good measure.
Therefore, someone has to decide what the organization wants the system to get better at. We also need to observe deployed AI in real-world conditions, because controlled pre-deployment testing cannot capture all the potentially unexpected consequences that could appear in actual use.
CEOs are currently asking questions such as “which model does it use?”, “How accurate is it?”, “How many agents can it run?” or “What does it cost per token?” But there’s another question that may matter much more: “What will this system be better at after working for us for twelve months?” Or, if you want to be more precise, “What does it learn from success and failure?”, or “If we replace the underlying model tomorrow, how much of what the system has learned about our company survives?” If the answer to the first question is “nothing unless engineers retrain or redesign it”, and the answer to the second one is “very little”, then the company may be renting intelligence without accumulating much intelligence of its own.
And I’m pretty sure that is not what you want. The next generation of corporate AI will not be defined simply by how intelligent it is on day one. It will be defined by how much better it becomes by day one-thousand. Make sure your company can capitalize on that, because others certainly will.