AI agents are rapidly becoming part of the modern DevOps toolkit. Imagine asking an AI assistant:“Deploy Orders Service version 2.3.1 to staging.”“Fetch logs for deployment job 1234.”“Restart the payment service in the development cluster.”“Show all failed deployments from the last 24 hours.”
Instead of navigating multiple dashboards, APIs, and command-line tools, engineers can interact with infrastructure using natural language.The productivity gains are impressive.
Building an agent that translates natural language into commands is easy.
However, there is a significant challenge. What happens when a user accidentally asks:“Deploy version 5.0 directly to production.” OR “Delete the production namespace.”
If the AI agent has direct access to infrastructure, a simple prompt could cause a major outage. The reality is that building the AI component is often easier than building the** governance, security, and control mechanisms**. required to operate safely in enterprise environments.
The key principle is simple: A DevOps AI agent should be treated as a planner, not as an executor.
Many early AI agent implementations follow a dangerous pattern:
User Request -> LLM -> Shell Command -> Production Environment.
Example: User: “Deploy Orders Service”
AI generates: kubectl apply -f deployment.yaml
The command is then executed automatically.
This approach creates multiple risks: Prompt injection attacks, Accidental destructive actions, Unauthorized production changes, Lack of auditability, and compliance violations.
An AI model should never be trusted with unrestricted execution privileges.
A safer architecture separates planning, governance, and execution. Here is the proposed approach.
In this model, the AI never interacts directly with infrastructur, i.e.,e Kubernetes, Jenkins, ArgoCD, Terraform, or cloud platforms.
Instead, it generates a structured action request.
The AI’s first responsibility is intent extraction.
User request: “Deploy Orders Service version 2.3.1 to staging.”
AI output:
{ “action”: “deploy_service”, “service”: “orders”, “version”: “2.3.1”, “environment”: “staging”}
Notice that no shell commands, API calls, or infrastructure details are involved.The AI’s job is simply to understand the user’s intent.
One of the most effective governance mechanisms is maintaining an explicit catalog of allowed actions.
For example:
ALLOWED_ACTIONS = [ “deploy_service”, “fetch_logs”, “restart_deployment”, “get_pod_status”]
If the AI produces:{ “action”: “delete_cluster”}
If that action does not exist in the catalog, the request is immediately rejected .This creates a boundary around what the agent can do.
Not all actions carry the same level of risk. A simple risk classification layer can help determine whether an action should be automatically executed or blocked.
View logs -> Low
Get pod status -> Low
Deploy to development -> Medium
Deploy to production -> High
By classifying actions, organizations can introduce additional approval steps for sensitive operations.
Policies should never be left to the discretion of the AI model.
ExamplesALLOW_ENVIRONMENTS = [ “dev”, “staging”]
Reject:if environment == “prod”: deny()
Allow:if action == “fetch_logs”: allow()
Before execution, generate a human-readable plan.
Example:
Example Execution Plan
Action:Deploy ServiceService:Orders API Version:2.3.1Environment:StagingDeployment Strategy:Rolling UpdateTarget Cluster:staging-cluster-01
This does two things: it makes the action transparent, and it gives a human a last chance to catch a mistake before it becomes an incident.
One of the most important design principles is avoiding AI-generated shell commands.
Avoid:
command = llm_responsesubprocess.run(command)
Instead, create approved tools.
DeployServiceTool()FetchLogsTool()RestartDeploymentTool()
The AI requests an action. The tool performs the action using predefined APIs.jenkins_client.start_job(…)
This approach is significantly safer and more predictable.
The agent shouldn’t get its own authorization model — it should inherit the one your organization already trusts. Map every request to the requesting user’s existing roles and permissions, and enforce those same boundaries at the tool layer, not just in the chat UI. If a user can’t deploy to the staging environment through the dashboard, the agent shouldn’t be able to do it on their behalf either.
Every request should be logged.
Example:
{ “user”: “john.doe”, “request”: “Deploy Orders Service”, “action”: “deploy_service”, “environment”: “staging”, “risk”: “medium”, “approved”: true, “result”: “success”}
Audit logs provide:Compliance support, Incident investigations, Change tracking, Security reviews
If an action cannot be audited, it should not be automated.
For teams building DevOps AI agents today:Agent Framework : LangGraph, OpenAI SDK, OllamaGovernance: Custom governance, Open Policy Agent (OPA)** Authentication**: existing enterprise authorization models ex: IBM App ID** Infrastructure Integration** : Jenkins APIs, ArgoCD APIs, Kubernetes APIs and Terraform APIsStorage: PostgreSQL, Cloudant
Here is the sample AI code intent generation and gavernence check .
import jsonimport uuidimport requestsfrom datetime import datetimeOLLAMA_URL = "http://localhost:11434"MODEL = "mistral"# ------------------------------------------------------------------# Governance Rules# ------------------------------------------------------------------ALLOWED_ACTIONS = [ "deploy_service", "restart_deployment", "fetch_logs", "get_pod_status"]ALLOWED_ENVIRONMENTS = [ "development", "staging"]ALLOWED_CLUSTERS = [ "dev-cluster-01", "stg-cluster-01"]# ------------------------------------------------------------------# Intent Extraction# ------------------------------------------------------------------def extract_intent(user_query, user_email, user_role): prompt = f"""You are a DevOps Intent Extractor.Convert the user request into JSON.Allowed actions:{', '.join(ALLOWED_ACTIONS)}Return ONLY valid JSON.Schema:{{ "action": "", "service": "", "environment": "", "cluster": ""}}Example:User:Deploy Orders Service version 2.3.1 to staging cluster stg-cluster-01Output:{{ "action": "deploy_service", "service": "orders", "version": "2.3.1", "environment": "staging", "cluster": "stg-cluster-01"}}User:{user_query}""" response = requests.post( f"{OLLAMA_URL}/api/generate", json={ "model": MODEL, "prompt": prompt, "format": "json", "stream": False }, timeout=60 ) response.raise_for_status() result = response.json() raw = result["response"].strip() try: llm_output = json.loads(raw) except json.JSONDecodeError: raise Exception( f"Model did not return valid JSON:\n{raw}" ) # Build enterprise-style intent object intent = { "intent_id": f"int-{uuid.uuid4().hex[:8]}", "timestamp": datetime.utcnow().isoformat(), "user": user_email, "role": user_role, "query": user_query, "action": llm_output.get("action", "").lower().strip(), "service": llm_output.get("service", ""), "environment": llm_output.get("environment", "").lower().strip(), "cluster": llm_output.get("cluster", "") } return intent# ------------------------------------------------------------------# Governance Engine# ------------------------------------------------------------------def policy_check(intent): # Rule 1: Allowed Action if intent["action"] not in ALLOWED_ACTIONS: raise Exception( f"Action '{intent['action']}' is not allowed" ) # Rule 2: Allowed Environment if intent["environment"] not in ALLOWED_ENVIRONMENTS: raise Exception( f"Environment '{intent['environment']}' is not approved" ) # Rule 3: Allowed Cluster if intent["cluster"] not in ALLOWED_CLUSTERS: raise Exception( f"Cluster '{intent['cluster']}' is not approved" ) return True# ------------------------------------------------------------------# Main# ------------------------------------------------------------------if __name__ == "__main__": query = ( "Deploy Orders Service version 2.3.1 " "to staging cluster stg-cluster-01" ) intent = extract_intent( user_query=query, user_email="john.doe@acme.com", user_role="developer" ) print("\nGenerated Intent") print(json.dumps(intent, indent=2)) policy_check(intent) print("\n Request Approved")
As organizations embrace AI-driven operations, the focus is on the intelligence of the agent. In reality, the most successful enterprise AI agents are not those with the most powerful models — they are the ones with the strongest governance.
A safe DevOps AI agent should:- Understand intent, not execute commands.- Operate within a predefined action catalog.- Follow policy-as-code rules.- Respect RBAC permissions.- Generate execution plans.- Use approved tools and APIs.- Maintain complete audit trails.
The future of AI in DevOps is not autonomous execution. It is governed by automation, where AI accelerates operations ensuring safety, compliance, and reliability.
Organizations that adopt this governance-first mindset will be able to leverage AI confidently without putting their production environments at risk.
Building Safe AI Agents for DevOps: Governance First, Automation Second was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.