Scaling AI Agents: A Step-by-Step Guide to Deploying ADK on GKE Autopilot

Google has released a step-by-step guide for deploying AI agents built with the Agent Development Kit (ADK) onto Google Kubernetes Engine (GKE) Autopilot, using Gemini on Vertex AI as the core model. The deployment process packages the ADK-based Python application as a Docker image, runs it as a GKE Autopilot Deployment, and secures communication with Vertex AI through Workload Identity. The guide also details exposing the agent externally using the Kubernetes Gateway API, providing developers with a managed infrastructure path for production-ready AI agents.

While building AI agents locally using Google’s Agent Development Kit ADK is an excellent way to prototype, production-ready agents require a robust, scalable infrastructure. For developers looking to move beyond simple instances and into the world of managed container orchestration, Google Kubernetes Engine GKE Autopilot offers the perfect balance of flexibility and ease of use. In this tutorial, I will walk you through building a technical agent with ADK and deploying it to GKE Autopilot. We will focus on utilizing Gemini on Vertex AI as the core model and ensure highest security standards by implementing Workload Identity for permission management. Deploying an ADK agent on GKE Autopilot involves more than just running a container. We leverage GKE's native capabilities to handle scaling and security. Our architecture consists of an ADK-based Python application packaged as a Docker image and stored in Artifact Registry. This container runs as a Deployment on GKE Autopilot, where it communicates securely with Vertex AI using Workload Identity—mapping a Kubernetes Service Account to a Google Cloud IAM Service Account. To expose the agent to the world, we use the Kubernetes Gateway API, the modern successor to Ingress, which provides a cleaner separation of concerns and native support for Google Cloud Load Balancing. Before we begin, ensure you have the following tools and accounts ready: uv for package management. gcloud installed and configured. kubectl command-line tool. jq for parsing JSON responses.Before interacting with Google Cloud services, you must authenticate your environment and set the active project. This ensures that both the gcloud CLI and your local Python environment can access Vertex AI. gcloud auth login gcloud config set project PROJECT ID gcloud auth application-default login export PROJECT ID=$ gcloud config get-value project export REGION=us-central1 export CLUSTER NAME=adk-cluster GKE Autopilot is the recommended way to run Kubernetes without managing nodes. It allows you to focus on your agent deployment while Google manages the infrastructure. Starting the cluster creation now allows it to provision in the background while we build the agent. gcloud container clusters create-auto $CLUSTER NAME --region $REGION While the cluster is provisioning, we can move on to building our agent. First, let's create our agent. Start by creating a folder for the agent code: mkdir adk-agent cd adk-agent Initialize a new Python project with uv: uv init Add dependencies uv add google-adk Create a new agent using the adk cli uv run adk create weather agent You will be asked to choose a model for the root agent. Choose gemini-2.5-flash Number 1 . Next you will be asked to choose a backend. Choose Vertex AI Number 2 . Next you will be asked to enter your Google Cloud project ID. Enter your project ID. Next you will be asked to enter your Google Cloud region. Choose a region of your choice. Example: us-central1 . The previous command scaffolded a new directory weather agent with the following structure: weather agent/ ├── .env ├── init .py └── agent.py ADK requires the agent code to be in agent.py file. Let's edit the agent.py file to add a simple tool for the agent. python from google.adk import Agent Define a simple tool for the agent def get weather city: str - str: """Returns the current weather in a city.""" return f"The weather in {city} is 90 degrees Fahrenheit and sunny." Initialize the agent with Vertex AI and Gemini root agent = Agent name="weather agent", model="gemini-2.5-pro", tools= get weather The agent.py file is the entry point for the agent. It is used to define the agent and its tools. The get weather function is a simple tool that returns the current weather in a city. For the purpose of this tutorial, we are using a hardcoded value for the weather. In a real-world scenario, you would use an API to get the current weather. Before deploying the agent to GKE Autopilot, we need to test it locally to ensure it works as expected. Run the following command to start the agent in debug mode with the web UI: uv run adk web Open http://localhost:8000 http://localhost:8000 in your browser and you should see the ADK web UI. You can then interact with your agent by typing messages in the chat interface. If the agent returns a message like "The weather in CITY is 90 degrees Fahrenheit and sunny." Congratulations your ADK agent is working. Now you can proceed to the next step. The ADK cli has a built-in command to deploy the agent to GKE Autopilot. However the default settings are not suitable for a production environment. For example, the default settings do not use Workload Identity for authentication with Vertex AI and to expose the Web UI via a Load Balancer on port 80. We will instead manage the lifecycle of the container ourselves. First we need to containerize the agent. Create a .dockerignore file in the adk-agent directory to prevent your local virtual environment from being copied into the image: .venv .adk pycache .pyc .env Create a Dockerfile for your agent in the adk-agent directory. We will use a multi-stage build to keep the final production image lightweight and secure: Stage 1: Build the virtual environment FROM python:3.10-slim AS builder Install uv COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/ Set working directory WORKDIR /app Force uv to use the system Python and use copy instead of symlinks ENV UV PYTHON PREFERENCE=only-system ENV UV LINK MODE=copy ENV UV COMPILE BYTECODE=1 ENV UV PYTHON=/usr/local/bin/python3 Install dependencies We copy only files needed for installation to maximize cache COPY pyproject.toml uv.lock ./ Note: We don't use --frozen yet as the host lock file might be slightly out of sync but sync will update it in the builder stage. RUN uv sync --no-install-project --no-dev --no-cache Copy the agent code COPY . . Sync the project itself RUN uv sync --no-dev --no-cache Stage 2: Runtime image FROM python:3.10-slim WORKDIR /app Copy the pre-built environment from the builder COPY --from=builder /app/.venv /app/.venv Copy the application code including weather agent folder COPY . . Add the environment to the PATH ENV PATH="/app/.venv/bin:$PATH" ENV PYTHONUNBUFFERED=1 Run the ADK API server We point to the weather agent folder CMD "adk", "api server", ".", "--host", "0.0.0.0", "--port", "8080" Build and push the image to Artifact Registry: Create repository gcloud artifacts repositories create adk-repo --repository-format=docker --location=$REGION Build and push gcloud builds submit --tag $REGION-docker.pkg.dev/$PROJECT ID/adk-repo/gke-agent:latest Security is paramount. Instead of hardcoding API keys, we use Workload Identity to grant the GKE pod permission to access Vertex AI. 1. Create an IAM Service Account : gcloud iam service-accounts create adk-gke-sa 2. Grant Vertex AI permissions : gcloud projects add-iam-policy-binding $PROJECT ID \ --member="serviceAccount:adk-gke-sa@$PROJECT ID.iam.gserviceaccount.com" \ --role="roles/aiplatform.user" 3. Allow the Kubernetes Service Account to impersonate the IAM SA : gcloud iam service-accounts add-iam-policy-binding adk-gke-sa@$PROJECT ID.iam.gserviceaccount.com \ --role="roles/iam.workloadIdentityUser" \ --member="serviceAccount:$PROJECT ID.svc.id.goog default/adk-ksa " Now, we define the Kubernetes resources. Create a deployment.yaml that includes the Service Account annotation for Workload Identity. Replace $PROJECT ID and $REGION with your actual project ID and region. apiVersion: v1 kind: ServiceAccount metadata: name: adk-ksa annotations: iam.gke.io/gcp-service-account: adk-gke-sa@$PROJECT ID.iam.gserviceaccount.com --- apiVersion: apps/v1 kind: Deployment metadata: name: adk-agent spec: replicas: 2 selector: matchLabels: app: adk-agent template: metadata: labels: app: adk-agent spec: serviceAccountName: adk-ksa containers: - name: adk-agent image: $REGION-docker.pkg.dev/$PROJECT ID/adk-repo/gke-agent:latest resources: requests: cpu: "500m" memory: "512Mi" limits: cpu: "1" memory: "1Gi" ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: adk-service spec: selector: app: adk-agent ports: - port: 80 targetPort: 8080 Apply the configuration: kubectl apply -f deployment.yaml Check the status of the deployment: kubectl get pods -w Once the pods are running, you can use kubectl port-forward to access the agent locally: kubectl port-forward svc/adk-service 8080:80 Since we deployed the agent without Web UI, we can't access it at http://localhost:8080 http://localhost:8080 . However, we can still interact with it using the API and curl . In a new terminal, run the following commands: Create a new session curl -X POST http://localhost:8080/apps/weather agent/users/u 123/sessions/s 123 Run a message curl -s -X POST http://localhost:8080/run \ -H "Content-Type: application/json" \ -d '{ "appName": "weather agent", "userId": "u 123", "sessionId": "s 123", "newMessage": { "role": "user", "parts": { "text": "Hey whats the weather in new york today" } } }' | jq . The curl command will return the response in JSON format. The jq command is used to parse the JSON response and display it in a more readable format. . You should see a response like: { "sessionId": "s 123", "messages": { "role": "assistant", "parts": { "text": "The weather in New York today is sunny with a high of 90 degrees Fahrenheit." } } } Finally, we expose the agent using the GKE Gateway API with a Google-managed TLS certificate. This is the recommended, production-grade approach — Google will automatically provision and renew the certificate for your domain. NB: GKE supports other options to provision certificates. You can use Let's Encrypt with cert-manager, pre-shared certificates, or any other certificate authority. You can check the GKE documentation https://docs.cloud.google.com/kubernetes-engine/docs/how-to/secure-gateway secure-using-ssl-certificate for more details. First, reserve a static IP address for your load balancer: gcloud compute addresses create adk-agent-ip --global export AGENT IP=$ gcloud compute addresses describe adk-agent-ip --global --format="value address " echo "Your IP: $AGENT IP" Point your domain's DNS A record at $AGENT IP . Example: adk.mydomain.com Create a Google-Managed Certificate. Replace adk.yourdomain.com with your actual domain:: gcloud compute ssl-certificates create adk-cert --domains adk.yourdomain.com --global Create a gateway.yaml with the following content: Gateway: HTTPS load balancer with the managed certificate and static IP apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: adk-gateway spec: gatewayClassName: gke-l7-global-external-managed listeners: - name: https protocol: HTTPS port: 443 tls: mode: Terminate options: networking.gke.io/pre-shared-certs: adk-cert addresses: - type: NamedAddress value: adk-agent-ip --- HTTPRoute: forward traffic to the ADK service apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: adk-route spec: parentRefs: - name: adk-gateway hostnames: - "api.yourdomain.com" rules: - backendRefs: - name: adk-service port: 80 --- apiVersion: networking.gke.io/v1 kind: HealthCheckPolicy metadata: name: adk-health namespace: default spec: default: checkIntervalSec: 15 timeoutSec: 5 healthyThreshold: 1 unhealthyThreshold: 2 logConfig: enabled: false config: type: HTTP httpHealthCheck: port: 8080 requestPath: /health targetRef: group: "" kind: Service name: adk-service Apply the configuration: kubectl apply -f gateway.yaml Certificate provisioning can take up to 20 minutes. Monitor the status with: gcloud compute ssl-certificates describe adk-cert --global Once the status shows Active , your agent is live at https://api.yourdomain.com . You can test it with: Create a new session curl -X POST https://api.yourdomain.com/apps/weather agent/users/u 124/sessions/s 124 Run a message curl -s -X POST https://api.yourdomain.com/run \ -H "Content-Type: application/json" \ -d '{ "appName": "weather agent", "userId": "u 124", "sessionId": "s 124", "newMessage": { "role": "user", "parts": { "text": "Hey whats the weather in new york today" } } }' | jq . By following these steps, you have successfully deployed a production-ready AI agent built with ADK onto GKE Autopilot that invokes Gemini on Vertex AI with Workload Identity for authentication. This setup ensures that your agent can scale horizontally to meet demand while maintaining a high security posture. As you look ahead, consider integrating more complex tools or leveraging GKE's multi-cluster capabilities for even greater resilience. For more details on the technologies used here, explore the official GKE documentation https://cloud.google.com/kubernetes-engine/docs and the ADK repository https://github.com/google/adk . To avoid ongoing charges, remember to delete the GKE cluster and the Artifact Registry repository when finished: kubectl delete -f gateway.yaml kubectl delete -f deployment.yaml gcloud compute addresses delete adk-agent-ip --global gcloud compute ssl-certificates delete adk-cert --global gcloud container clusters delete $CLUSTER NAME --region $REGION gcloud artifacts repositories delete adk-repo --location $REGION