AI coding tools like GitHub Copilot, Gemini Code Assist and Cursor have vastly increased the speed of coding for AI projects, making at least that part of project development seem like a bullet train. But that train stops short of the station. That “last mile” of the development process is where the truly hard work begins — and where many projects are derailed.
Nearly eight in 10 companies report using gen AI to some extent, yet about 90 percent of vertical, function-specific AI use cases remain stuck in pilot mode, according to McKinsey. More than 80 percent of companies also report no material contribution to earnings from their gen AI initiatives.
Finding similarly grim levels of success, IDC authors conclude:
“The high number of AI POCs [proofs of concept] but low conversion to production indicates the low level of organizational readiness in terms of data, processes and IT infrastructure.”
AI tools are exceptionally fast at helping developers spin up working prototypes and generate massive amounts of code, sometimes in just minutes. But the familiar 80/20 problem often applies: Generating the bulk of the code can be fast, while the final production work consumes a disproportionate amount of the effort. This “last mile” requires rigorous software engineering to ensure reliability, scalability and security.
More, Faster Code Does Not Equal AI Success
Even if the project has a well-defined business use case, the lack of which RAND found to be one of the leading causes of project failure, other organizational issues arise. Who will own the mounds of code created? Will a data team create a model, then throw it over the fence to an engineering team to maintain? Data and model drift are ongoing problems. Who will address that? What is the cost to maintain the code? How will you secure it?
Along with mountains of code, AI can produce significantly more technical debt. GitClear’s analysis of 211 million lines of code found:
-
The prevalence of duplicate code blocks increased sharply, with the report describing four times more code cloning.
-
Copy-pasted lines rose from 8.3 percent in 2021 to 12.3 percent in 2024.
-
The share of changed code associated with refactoring fell from 25 percent in 2021 to less than 10 percent in 2024.
-
Short-term code churn also increased. CodeRabbit research comparing AI- and human-written code found both contain similar errors, but AI-generated code contains them more frequently. AI pull requests contain 1.4 times more critical errors and 1.7 times more major errors.
Logic errors such as business logic mistakes, incorrect dependencies, flawed control flow and misconfigurations were 75 percent more common in AI code. These are the most expensive to fix and create downstream headaches.
The research suggests companies provide the context and guardrails around their data to ensure all code entering production is safe, maintainable and correct.
“Our analysis shows that AI-generated code is consistently more variable, more error-prone, and more likely to introduce high-severity issues without the right protections in place,” the authors concluded.
While your developers are dealing with thousands of lines of AI-generated code, attackers are already using autonomous agents to detect flaws at similar speed. Once in production, AI systems are susceptible to unique vectors of attack, such as data poisoning, prompt injection of LLMs or attacks designed to trick the model.
A lack of planning for the ongoing iteration of AI models and data can lead to outdated models that fail to achieve the project’s objectives.
Just as a car maker must plan and test in the factory for all scenarios that the buyer might encounter, the architecture has to support the performance in real operating conditions.
Ensuring Scalability, Reliability, Security
In a classic paper, Google noted that only a tiny fraction of a production AI system is the actual machine learning code. The rest is a massive plumbing problem.
There are problems of scalability. A model that runs smoothly in a demo can stall when hit with thousands of concurrent, low-latency requests from real users.
It might run great on a greenfield architecture, but most use cases involve integration with existing systems. That integration can become a Gordian knot when dealing with legacy code, service boundaries, authorization middleware, protobuf schemas and more.
Then there’s the “Dory problem.” Like the character in “Finding Nemo,” large language models do not inherently retain persistent context from one interaction to the next. Debugging can require understanding massive, complex architectures built over decades — understanding that often resides only in engineers’ heads — and knowledge of what already exists, what was done before and perhaps didn’t work.
Similarly, AI doesn’t innately understand what your company’s data means. In a 2026 Cloudera/Harvard Business Review survey, only 7 percent of enterprises reported their data is completely ready for AI.
Some companies are addressing this problem with a production-grade governed, tool-agnostic semantic layer that enforces a single set of business definitions across every BI tool, AI agent and application.
This centralized framework provides a single, consistent definition for key metrics and business logic, spells out relationships between data elements and embeds metadata for the data in your AI system.
AI relies on continuous data pipelines, which can be fragile. Connecting data silos can be a headache. And making a change in an upstream data source can cause the model to break, often silently.
Observability poses other challenges. Standard observability solutions focus on performance issues such as CPU usage or latency yet struggle to effectively track and manage the nuances of AI behavior. An AI system may appear to perform correctly, but its output is questionable, especially its tendency to hallucinate, or simply make stuff up. It might leak sensitive or proprietary information, repeat bias in its training data or violate compliance regulations.
“The hardest part of AI is not generating outputs. It is designing systems people can trust once they are live,” Sai Kumar Reddy Midde, senior programmer at Kantar and founder and CMO at Growthora Media, posted on a LinkedIn discussion of crossing that “last mile.”
Jim Woods, CEO of Seattle Consulting Group, adds: “AI transformation rarely fails because of the technology itself. More often it stalls because the operating model around the technology remains unchanged. AI can generate insight at extraordinary speed, but most enterprises still rely on decision structures designed for a slower information cycle — layered approvals, fragmented ownership, and incentives that favor stability over experimentation. When those conditions persist, the bottleneck shifts from the technology to the organization. The last mile becomes less a technical challenge and more an organizational design problem.”