{"slug": "the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai", "title": "The real LLMOps risk isn't the model. It's shadow AI.", "summary": "CNCF's Daniel Bryant argues that LLMOps should be integrated into existing platform engineering rather than treated as a separate stack, warning that shadow AI pipelines built outside governance pose a greater risk than model hallucinations. The CNCF Platforms Whitepaper frames model fine-tuning, vector databases, prompt registries, and inference endpoints as standard platform capabilities requiring unified API, versioning, and ownership.", "body_md": "Large language models broke the clean \"train it, test it, ship it\" model of production ML. The thing being operated is now a system that chains prompts, queries vector databases, and produces output judged on tone and safety — not just accuracy. That's LLMOps. And it's currently landing on top of your existing DevOps and MLOps workflows without a clear owner.\n\nCNCF's Daniel Bryant has a clear take on who should own it — and the argument is sharper than it sounds.\n\n\"LLMOps doesn't need its own kingdom. It needs a well-run platform willing to let it in.\"\n\nLLMOps isn't just MLOps with a new label. The gap is real:\n\nMLOps teams have already built a parallel stack (MLflow, Kubeflow, Weights & Biases) because DevOps tooling never anticipated data versioning or drift monitoring. Without intervention, LLMOps becomes a *third* parallel stack, invisible to whoever governs the rest.\n\nThe bigger operational risk isn't a hallucinating chatbot. It's a team standing up its own RAG pipeline against an unreviewed vector store, outside any platform governance. Same pattern that made the DevOps-versus-platform split painful: a capability gets built outside the platform because the platform wasn't ready, and it never gets folded back in.\n\nBryant's framing via the CNCF Platforms Whitepaper is clean: model fine-tuning jobs, vector databases, prompt registries, and inference endpoints are just another platform capability. They need the same API, versioning, and ownership as anything else.\n\nThe tooling already exists in the CNCF ecosystem — Backstage at the product layer, Crossplane at the infrastructure layer, Kratix/KubeVela/KusionStack in the middle, exposing LLM pipelines through the same self-service interface as everything else.\n\n**If you're a platform engineer:**\n\n**If you're on an MLOps or AI team:**\n\n**If you're in platform leadership:**\n\nThe CNCF TAG App Delivery Platforms Working Group is actively working on this. If your org is sorting out LLMOps ownership, it's worth following.\n\n[Source: CNCF Blog — LLMOps and platform engineering: Who should own the AI pipeline?](https://www.cncf.io/blog/2026/08/13/llmops-and-platform-engineering-who-should-own-the-ai-pipeline/)\n\n*✏️ Drafted with KewBot (AI), edited and approved by Drew.*", "url": "https://wpnews.pro/news/the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai", "canonical_source": "https://dev.to/thegatewayguy/the-real-llmops-risk-isnt-the-model-its-shadow-ai-4m86", "published_at": "2026-08-14 07:56:32+00:00", "updated_at": "2026-08-14 08:15:46.409807+00:00", "lang": "en", "topics": ["mlops", "ai-infrastructure", "ai-safety", "developer-tools"], "entities": ["CNCF", "Daniel Bryant", "MLflow", "Kubeflow", "Weights & Biases", "Backstage", "Crossplane", "Kratix"], "alternates": {"html": "https://wpnews.pro/news/the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai", "markdown": "https://wpnews.pro/news/the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai.md", "text": "https://wpnews.pro/news/the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai.txt", "jsonld": "https://wpnews.pro/news/the-real-llmops-risk-isn-t-the-model-it-s-shadow-ai.jsonld"}}