Member-only story
It is usually not the model. It is the happy-path harness around the model: latency, retries, queues, stale retrieval, and the total lack of visibility when things start going wrong. #
Read the article for free
[.]here
The bug report was straightforward. The internal copilot recommended running kubectl delete namespace production
.
The model had not hallucinated. It accurately summarized an year old operational runbook that was still active in the vector database. A developer asked a question about clearing a stuck deployment, the retrieval system pulled the highest-scoring semantic match, and the language model formatted the command into a clean code block.
When an AI application hits its first thousand real users, the architecture you built for staging stops working. Real traffic exposes the operational harness around the model. The prototype assumed clean prompts, low concurrency, and successful tool calls. Production usage introduces latency spikes, retry storms, stale ingestion pipelines, and a complete lack of visibility when things start failing silently.