Scaling stateful Gemini 4 Argon agent sessions on GKE: Connection draining, startup boost, and session persistence

Most Kubernetes architectures are built around a comfortable assumption: HTTP requests arrive, execute stateless logic in under 200 milliseconds, and return a response. Multi-turn Gemini 4 Argon agent workflows invert every part of that profile. A single user request can initiate a 3-minute bidirectional streaming session where the agent plans, executes external tools, ingests multi-megabyte documents into its context window, and streams intermediate reasoning steps back to the client.

When you deploy these long-running, stateful agent orchestrators onto Google Kubernetes Engine (GKE) using default web-service configurations, three production failure modes appear almost immediately:

SIGTERM shutdowns sever active WebSocket or Server-Sent Events (SSE) streams mid-turn when GKE scales down or rolls out a new image. In this article, I will show you how to architect GKE for stateful Gemini 4 Argon agent runtimes by combining VPA CPU Startup Boost, long-horizon connection draining via GCPBackendPolicy, and external state checkpointing.

An agent runtime Pod traverses three distinct operational phases across its lifecycle:

readinessProbe, GKE uses In-Place Pod Resize (IPPR) to step CPU down to baseline without restarting the container.preStop hook and a 300-second termination grace period allow active reasoning loops to finish and flush final checkpoints before the container exits. Each GKE primitive addresses a specific stage of this lifecycle:

Agent orchestrator images frequently bundle the google-genai SDK, gRPC C-extensions, Pydantic schema validators, and document parsing libraries. If you size your container's baseline CPU request for steady-state I/O waiting (where the Pod spends most of its time awaiting network streams from the Gemini platform), the container only needs 500m CPU at steady state, but requires 1500m CPU during the first 10 seconds of startup.

Using VPA CPU startup boost (available in Public Preview on GKE 1.36.0-gke.4447000 and later in the Rapid Channel), GKE temporarily boosts the container's CPU request at admission and uses Kubernetes In-Place Pod Resize (IPPR) to lower the CPU allocation back to 500m without restarting the container once it passes its readinessProbe.

Performance ComputeClass. Create a VerticalPodAutoscaler policy named agent-runtime-startup-boost.yaml that triples the baseline 500m CPU allocation to 1500m during boot and unboosts immediately (durationSeconds: 0) upon readiness. Setting updateMode: "Off" applies the startup boost and in-place downscale without enabling continuous VPA actuation, avoiding conflicts with Horizontal Pod Autoscaler (HPA):

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: gemini-agent-startup-boost
  namespace: agent-runtime
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: gemini-session-orchestrator
  updatePolicy:
    updateMode: "Off"
  startupBoost:
    cpu:
      type: Factor
      factor: 3
      durationSeconds: 0

To prevent GKE from severing active Gemini 4 Argon streaming sessions during a rolling deployment, you need to coordinate timeouts across two layers: the Kubernetes Pod lifecycle (terminationGracePeriodSeconds and preStop hooks) and the GKE Gateway load balancer (GCPBackendPolicy).

Save the following manifest as agent-session-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gemini-session-orchestrator
  namespace: agent-runtime
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0
  selector:
    matchLabels:
      app: gemini-session-orchestrator
  template:
    metadata:
      labels:
        app: gemini-session-orchestrator
    spec:
      serviceAccountName: gemini-agent-sa
      terminationGracePeriodSeconds: 300
      containers:
      - name: agent-server
        image: us-central1-docker.pkg.dev/production-ai-platform/agents/gemini-4-session-server:v1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: GOOGLE_CLOUD_PROJECT
          value: "production-ai-platform"
        - name: REDIS_HOST
          value: "10.128.0.45"
        resources:
          requests:
            cpu: "500m"
            memory: "2Gi"
          limits:
            memory: "2Gi"
        readinessProbe:
          httpGet:
            path: /healthz/ready
            port: 8080
          periodSeconds: 5
          failureThreshold: 2
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 10"]
---
apiVersion: v1
kind: Service
metadata:
  name: gemini-session-svc
  namespace: agent-runtime
spec:
  selector:
    app: gemini-session-orchestrator
  ports:
  - port: 80
    targetPort: 8080
---
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
  name: gemini-streaming-backend-policy
  namespace: agent-runtime
spec:
  default:
    timeoutSec: 600
    connectionDraining:
      drainingTimeoutSec: 300
    sessionAffinity:
      type: GENERATED_COOKIE
      cookieTtlSec: 3600
  targetRef:
    group: ""
    kind: Service
    name: gemini-session-svc

Notice three critical settings in this configuration:

maxUnavailable: 0`` Ready state before terminating any existing Pod holding active sessions.sleep 10) timeoutSec: 600 and drainingTimeoutSec: 300 Even with a 300-second drain window, an agent session might span multiple user interactions over an hour, or a node preemption on Spot capacity might give only 30 seconds of warning. By checkpointing the conversation history after every Gemini 4 Argon turn into Redis (such as Memorystore for Redis), any replica in the Deployment can transparently resume the session if the original Pod terminates.

Here is the core session server (session_server.py) showing graceful SIGTERM tracking and per-turn Redis checkpointing with the google-genai SDK:

import asyncio
import json
import os
import signal
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
import redis.asyncio as redis
from google import genai
from google.genai import types

draining = False
active_turns = 0

class TurnRequest(BaseModel):
    user_message: str

def handle_sigterm(*_args):
    global draining
    draining = True

@asynccontextmanager
async def lifespan(_app: FastAPI):
    signal.signal(signal.SIGTERM, handle_sigterm)
    yield
    while active_turns > 0:
        await asyncio.sleep(1)
    await redis_client.aclose()

app = FastAPI(lifespan=lifespan)
genai_client = genai.Client(
    vertexai=True,
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    location=os.environ.get("VERTEX_LOCATION", "us-central1"),
)
redis_client = redis.Redis(host=os.environ["REDIS_HOST"], port=6379, decode_responses=True)
MODEL_ID = os.environ.get("GEMINI_MODEL", "gemini-4-argon")

@app.get("/healthz/ready")
async def readiness_check():
    if draining:
        raise HTTPException(status_code=503, detail="Pod is draining active sessions")
    return {"status": "ready"}

@app.post("/v1/sessions/{session_id}/stream")
async def stream_agent_turn(session_id: str, request: TurnRequest):
    raw_history = await redis_client.get(f"session:{session_id}")
    history = json.loads(raw_history) if raw_history else []
    history.append({"role": "user", "parts": [{"text": request.user_message}]})

    contents = [
        types.Content(role=turn["role"], parts=[types.Part.from_text(text=p["text"]) for p in turn["parts"]])
        for turn in history
    ]

    async def event_generator():
        global active_turns
        active_turns += 1
        assembled_response = []
        try:
            async for chunk in await genai_client.aio.models.generate_content_stream(
                model=MODEL_ID,
                contents=contents,
            ):
                if chunk.text:
                    assembled_response.append(chunk.text)
                    yield f"data: {json.dumps({'text': chunk.text})}\n\n"

            history.append({"role": "model", "parts": [{"text": "".join(assembled_response)}]})
            await redis_client.set(f"session:{session_id}", json.dumps(history), ex=86400)
        finally:
            active_turns -= 1

    return StreamingResponse(event_generator(), media_type="text/event-stream")

Once deployed, confirm that VPA CPU startup boost is accelerating Pod initialization and resizing CPU in place without restarting the container:

kubectl get pods -n agent-runtime \
    -l app=gemini-session-orchestrator \
    -o custom-columns="NAME:.metadata.name,CPU_REQ:.spec.containers[0].resources.requests.cpu,BOOST_ANNOTATION:.metadata.annotations.vpaCpuStartupBoost\/agent-server"

kubectl get events -n agent-runtime --field-selector reason=InPlaceResizedByVPA

Expected event output:

LAST SEEN   TYPE     REASON                 OBJECT                                             MESSAGE
14s         Normal   InPlaceResizedByVPA    pod/gemini-session-orchestrator-6b8f9c7d4-k29lm    Pod was in-place resized by VPA Updater (CPU: 1500m -> 500m)

To verify that rolling updates drain cleanly without dropping streams, trigger a rollout while streaming a multi-step Gemini 4 Argon response and monitor active backend connections in Cloud Monitoring using the metric loadbalancing.googleapis.com/https/backend_request_count alongside kubernetes.io/container/cpu/core_usage_time.

memory requests and limits to accommodate the largest concurrent payload your agent processes.spot: true in a GKE Running Gemini 4 Argon agents reliably in production requires adapting Kubernetes from short-lived request/response defaults to stateful, long-horizon execution. By combining VPA CPU Startup Boost for fast initialization, GCPBackendPolicy for long streaming timeouts, and per-turn state checkpointing, your agent deployments can scale and roll out new versions without interrupting active users.

To dive deeper into these GKE capabilities, read the VPA CPU Startup Boost guide, review the GKE Gateway GCPBackendPolicy reference, and check the Gemini API documentation on Google Cloud.