# Enterprise AI's Reliability Gap: When Models Flop in the Real World > Source: > Published: 2026-07-16 08:54:07+00:00 # Enterprise AI's Reliability Gap: When Models Flop in the Real World Only 5% of enterprises trust AI agents in production despite 85% piloting them. Amazon's Bryan Silverthorn outlines why reliability, not benchmarks, defines success. The enterprise AI sector faces a stark reality: 85% of enterprises are piloting AI agents, yet a mere 5% trust them in production. This gap isn't just about refining models, it's about understanding and measuring what truly matters. At VB Transform 2026, Bryan Silverthorn from Amazon's [AGI](/glossary/agi) lab emphasized that bridging this gap requires more than just better benchmarks. ## Decoding [AI Agent](/glossary/ai-agent) Reliability Silverthorn, who guides [multimodal](/glossary/multimodal) agent [training](/glossary/training) at Amazon, suggests a framework that dissects reliability into consistency, predictability, robustness, and safety. This approach, rooted in Princeton research, untangles the complexities often obscured in evaluations. The problem? AI agents often excel in controlled environments but falter when facing real-world applications. Consider a case where an agent flawlessly handled software QA for two months, only to collapse due to a minor software change. The vision [encoder](/glossary/encoder) misinterpreted serial numbers based on their screen location, a small shift, yet a critical failure. The takeaway isn't just about better models. it's about rigorous measurement aligned with application stakes. ## Amazon's 'Intern' Philosophy Silverthorn's perspective isn't just technical, it's cultural. Amazon mocks its agents as 'interns' to illustrate their potential and fallibility. Like interns, these AI agents can perform great feats or derail spectacularly. Managing these 'interns' isn't just about software prowess but involves strategic risk assessment and mitigation. In Amazon's AGI lab, agents occasionally make mistakes, but this trade-off is accepted for research velocity. Imagine an agent running experiments 24/7, autonomously executing high-level research plans. This philosophy underscores the need for adaptive management rather than rigid control. ## Beyond Pilot Purgatory For enterprises trapped in pilot limbo, the solution isn't smarter agents, it's smarter management. AI's self-improvement remains a distant dream, and reliance solely on computer use is obsolete. Future agents will integrate with APIs, MCP, and more. The goal isn't to ask if an agent can succeed once, but if it can do so consistently, a thousand times over. Enterprises breaking free from the 85% ceiling won't boast the most intelligent agents, but the most effective managers. If the AI can hold a wallet, who writes the risk model? The intersection is real. Ninety percent of the projects aren't. But the ones that succeed? They'll define the industry. Get AI news in your inbox Daily digest of what matters in AI. ## Key Terms Explained [AGI](/glossary/agi) Artificial General Intelligence. [AI Agent](/glossary/ai-agent) An autonomous AI system that can perceive its environment, make decisions, and take actions to achieve goals. [Encoder](/glossary/encoder) The part of a neural network that processes input data into an internal representation. [MCP](/glossary/mcp) Model Context Protocol (MCP) is an open standard created by Anthropic that lets AI models connect to external tools, data sources, and APIs through a unified interface.