Is your company’s AI getting smarter every day? It should be

Corporate AI deployments largely fail to learn from operational experience, according to a Fast Company analysis that distinguishes memory accumulation from genuine learning and cites Microsoft guidance that gains come from feeding production signals into prompts, routing, and retrieval. The piece invokes Microsoft CEO Satya Nadella's distinction between a replaceable general model and enduring "company veteran" expertise, and points to the research field of continual learning, which aims to let intelligent systems learn incrementally from changing streams of experience while retaining prior knowledge.

Imagine working with someone who was completely unable to learn, to gain any experience whatsoever. There’s no need to picture someone completely stupid: just think of someone who joins the company, starts to deal with a lot of customers, experiences successes and failures, attends meetings, makes decisions . . . but one year later, has the same skills and capabilities as on day one. Would you be able to call that person experienced? Surprisingly, that’s exactly what happens with much of the corporate AI https://www.fastcompany.com/91595338/corporate-america-is-embracing-ai-slower-than-the-hype-suggests-but-the-pace-is-increasing that is being deployed and implemented these days. Obviously, I’m not saying that an AI https://www.fastcompany.com/section/artificial-intelligence system cannot change after deployment: we all know it can, we have experienced that even in our own interactions with chatbots. Systems can accumulate memory, retrieve previous interactions, be updated, fine-tuned or have their surrounding logic modified. However, remembering is not the same as learning. A system can accumulate information without becoming better at deciding what to do next. Using an LLM more does not, by itself, make the deployed model learn from the consequences of its actions. Satya Nadella saw that clearly https://www.fastcompany.com/91561371/satya-nadella-is-asking-the-right-ai-question , and distinguished the replaceable general model from the enduring “company veteran” expertise that ought to accumulate around it. How many streams of lessons does your company produce every day? From a disgruntled customer, to a discount that worked or didn’t work, a supplier that delivered late, a support operation that solved a problem or made it worse, a sales approach that failed in a segment but worked on a different one . . . All these things are not simply “data”: they are specific actions, followed by specific consequences. How much of that experience can make tomorrow’s AI better https://www.fastcompany.com/91376643/the-future-of-generative-ai-from-hype-to-hypergrowth-designing-tomorrows-organization than today’s? If we take into account what Microsoft has to say about this https://urldefense.proofpoint.com/v2/url?u=https-3A learn.microsoft.com en-2Dus startups build ai ai-2Dapp-2Darchitecture&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=a56VqYcYm0gaftoH3FHYT4a 9xUoRoai2bX2v20D4sg&e= , the possible gains come when you learn what happened in production and you are able to feed those signals into prompts, routing, retrieval and other system decisions: everyday operations need to become lessons from which your system can learn. Imagine a team of very disciplined salespeople that keep a notebook with their activity carefully annotated: after thousands of calls, the notebook will contain an awful lot of information, but the judgment of the salespeople will be pretty much the same. Have they really learned anything? Remembering that one specific customer rejected a 10% discount can be useful or practical, but learning that this particular customer segment responds better to faster implementations than to discounts is something qualitatively different. There’s even a research field called continual learning https://urldefense.proofpoint.com/v2/url?u=https-3A www.sciencedirect.com science article pii S0893608025011074&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=qfFbrPE-ZsZbxu91foONn0w3SvoR8-CT5G7AuNKSK2w&e= , whose goal is to allow intelligent systems to learn incrementally from ever-changing streams of experience while retaining useful previous knowledge: basically, how to learn new things without forgetting old ones. When different competitors can buy the same frontier models https://www.fastcompany.com/91577359/why-u-s-ai-companies-cant-match-chinas-open-weight-frontier-models , access to these models does not constitute a sustainable competitive advantage anymore. What can become the essence of such an advantage is actually all the things you have been able to learn from your particular history: customers, decisions, campaigns, corrections, failures, exceptions, outcomes, etc. Satya Nadella proposed an interesting “company veteran test” in this regard: imagine that tomorrow you replace a model with a different one, will the expertise accumulated inside your organization remain? The real opportunity lies in building a learning loop in which human and token capital can compound, rather than simply choosing the best model https://www.fastcompany.com/91367690/what-top-ai-models-really-think-each-other-survey-opinion-llms-chatgpt-claude-gemini-deepseek-llama-grok . Accumulated learning should persist outside the replaceable model rather than disappearing when you get a new one. The idea is that scale should make the company smarter, not just bigger. Traditional software can process a million transactions instead of ten thousand. An intelligent system should go one step further: interaction number one million should leave it better prepared for interaction 1,000,001. Compounding intelligence is what you should get from your corporate system, instead of just databases, records or memory. Recommendation systems at companies such as YouTube, Spotify or Amazon have long used streams of user behavior and feedback to improve future recommendations https://urldefense.proofpoint.com/v2/url?u=https-3A research.google pubs learning-2Dfrom-2Dequivalence-2Dqueries-2Drevisited &d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=3TgYEex5np2Xk8K MYYhb8dbknVQCiNFFGQRuVPnwVI&e= or at the very least, try to improve them , and they do so even without AI. Why should we accept corporate AI https://www.fastcompany.com/91564462/how-ai-reshapes-the-corporate-tech-stack that doesn’t learn and improve from corporate experience? Feedback loops can be bad as well as good https://urldefense.proofpoint.com/v2/url?u=https-3A developers.google.com machine-2Dlearning crash-2Dcourse production-2Dml-2Dsystems questions&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=PRzPAnB1rtgUKHqgM6CfBQ19zRvRNUnynAEdWp kz4E&e= . The company needs to decide what success actually means. Faster customer service calls are not necessarily better if the customers leave afterwards because they didn’t like the quality of the interaction. A higher conversion is not necessarily better if discounts are eroding our margins. More output is not better if, as a consequence, the quality gets worse. The British economist Charles Goodhart https://urldefense.proofpoint.com/v2/url?u=https-3A en.wikipedia.org wiki Charles-5FGoodhart&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=1F6EbMvKXzfZY8oz IYzQpstmTHjWPsmaRFqyhxfoJU&e= has a quote I use in all my courses: when a measure becomes a target, it ceases to be a good measure. Therefore, someone has to decide what the organization wants the system to get better at. We also need to observe deployed AI in real-world conditions https://urldefense.proofpoint.com/v2/url?u=https-3A www.nist.gov publications challenges-2Dmonitoring-2Ddeployed-2Dai-2Dsystems-2Dcenter-2Dai-2Dstandards-2Dand-2Dinnovation&d=DwMFaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A CdpgnVfiiMM&r=xHenyQfyc6YcuCNMBsOvfYGQILM1d1ruredVZikn4HE&m=k -R3-tpv3SjXoQewvnXAGAZIjl-RbdF Ydf0-sGodccMTTK2TGas9rszQJCkNuj&s=JVHRJqZpupI7O- mbL705-yZPL2YdjVqYxIwdgMVkGI&e= , because controlled pre-deployment testing cannot capture all the potentially unexpected consequences that could appear in actual use. CEOs are currently asking questions such as “which model does it use?”, “How accurate is it?”, “How many agents can it run?” or “What does it cost per token?” But there’s another question that may matter much more: “What will this system be better at after working for us for twelve months?” Or, if you want to be more precise, “What does it learn from success and failure?”, or “If we replace the underlying model tomorrow, how much of what the system has learned about our company survives?” If the answer to the first question is “nothing unless engineers retrain or redesign it”, and the answer to the second one is “very little”, then the company may be renting intelligence without accumulating much intelligence of its own. And I’m pretty sure that is not what you want. The next generation of corporate AI will not be defined simply by how intelligent it is on day one. It will be defined by how much better it becomes by day one-thousand. Make sure your company can capitalize on that, because others certainly will.