# Scaling stateful Gemini 4 Argon agent sessions on GKE: Connection draining, startup boost, and session persistence

> Source: <https://dev.to/googlecloud/scaling-stateful-gemini-4-argon-agent-sessions-on-gke-connection-draining-startup-boost-and-13ln>
> Published: 2026-10-22 16:05:20+00:00

Most Kubernetes architectures are built around a comfortable assumption: HTTP requests arrive, execute stateless logic in under 200 milliseconds, and return a response. Multi-turn [Gemini 4 Argon](https://cloud.google.com/gemini-enterprise-agent-platform/models/gemini/4-argon) agent workflows invert every part of that profile. A single user request can initiate a 3-minute bidirectional streaming session where the agent plans, executes external tools, ingests multi-megabyte documents into its context window, and streams intermediate reasoning steps back to the client.

When you deploy these long-running, stateful agent orchestrators onto [Google Kubernetes Engine (GKE)](https://cloud.google.com/kubernetes-engine) using default web-service configurations, three production failure modes appear almost immediately:

`SIGTERM` shutdowns sever active WebSocket or Server-Sent Events (SSE) streams mid-turn when GKE scales down or rolls out a new image.
In this article, I will show you how to architect GKE for stateful Gemini 4 Argon agent runtimes by combining [VPA CPU Startup Boost](https://cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup), long-horizon connection draining via `GCPBackendPolicy`, and external state checkpointing.

An agent runtime Pod traverses three distinct operational phases across its lifecycle:

`readinessProbe`, GKE uses In-Place Pod Resize (IPPR) to step CPU down to baseline without restarting the container.`preStop` hook and a 300-second termination grace period allow active reasoning loops to finish and flush final checkpoints before the container exits.
Each GKE primitive addresses a specific stage of this lifecycle:

| Operational challenge | Default Kubernetes behavior | GKE production pattern for Gemini 4 Argon agents | 
|---|---|---|
| **Heavy SDK initialization** | Container throttled by baseline CFS CPU quota during boot | **VPA CPU startup boost** (`updateMode: "Off"` ) grants temporary CPU headroom and resizes down live via IPPR after readiness | 
| **Long streaming sessions** | Load balancer closes connection at 30 seconds | **`GCPBackendPolicy`** sets` timeoutSec: 600` and`drainingTimeoutSec: 300` with cookie-based session affinity | 
| **Pod termination during rollout** | `SIGTERM` followed by`SIGKILL` after 30 seconds | **`terminationGracePeriodSeconds: 300`** paired with a graceful drain handler that completes active Gemini 4 Argon turns and checkpoints state | 

Agent orchestrator images frequently bundle the `google-genai` SDK, gRPC C-extensions, Pydantic schema validators, and document parsing libraries. If you size your container's baseline CPU request for steady-state I/O waiting (where the Pod spends most of its time awaiting network streams from the Gemini platform), the container only needs `500m` CPU at steady state, but requires `1500m` CPU during the first 10 seconds of startup.

Using **VPA CPU startup boost** (available in Public Preview on GKE `1.36.0-gke.4447000` and later in the Rapid Channel), GKE temporarily boosts the container's CPU request at admission and uses Kubernetes In-Place Pod Resize (IPPR) to lower the CPU allocation back to `500m` without restarting the container once it passes its `readinessProbe`.

`Performance` `ComputeClass`.
Create a `VerticalPodAutoscaler` policy named `agent-runtime-startup-boost.yaml` that triples the baseline `500m` CPU allocation to `1500m` during boot and unboosts immediately (`durationSeconds: 0`) upon readiness. Setting `updateMode: "Off"` applies the startup boost and in-place downscale without enabling continuous VPA actuation, avoiding conflicts with Horizontal Pod Autoscaler (HPA):

```
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: gemini-agent-startup-boost
  namespace: agent-runtime
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: gemini-session-orchestrator
  updatePolicy:
    updateMode: "Off"
  startupBoost:
    cpu:
      type: Factor
      factor: 3
      durationSeconds: 0
```

To prevent GKE from severing active Gemini 4 Argon streaming sessions during a rolling deployment, you need to coordinate timeouts across two layers: the Kubernetes Pod lifecycle (`terminationGracePeriodSeconds` and `preStop` hooks) and the GKE Gateway load balancer (`GCPBackendPolicy`).

Save the following manifest as `agent-session-deployment.yaml`:

```
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gemini-session-orchestrator
  namespace: agent-runtime
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 0
  selector:
    matchLabels:
      app: gemini-session-orchestrator
  template:
    metadata:
      labels:
        app: gemini-session-orchestrator
    spec:
      serviceAccountName: gemini-agent-sa
      terminationGracePeriodSeconds: 300
      containers:
      - name: agent-server
        image: us-central1-docker.pkg.dev/production-ai-platform/agents/gemini-4-session-server:v1.0.0
        ports:
        - containerPort: 8080
        env:
        - name: GOOGLE_CLOUD_PROJECT
          value: "production-ai-platform"
        - name: REDIS_HOST
          value: "10.128.0.45"
        resources:
          requests:
            cpu: "500m"
            memory: "2Gi"
          limits:
            memory: "2Gi"
        readinessProbe:
          httpGet:
            path: /healthz/ready
            port: 8080
          periodSeconds: 5
          failureThreshold: 2
        lifecycle:
          preStop:
            exec:
              command: ["/bin/sh", "-c", "sleep 10"]
---
apiVersion: v1
kind: Service
metadata:
  name: gemini-session-svc
  namespace: agent-runtime
spec:
  selector:
    app: gemini-session-orchestrator
  ports:
  - port: 80
    targetPort: 8080
---
apiVersion: networking.gke.io/v1
kind: GCPBackendPolicy
metadata:
  name: gemini-streaming-backend-policy
  namespace: agent-runtime
spec:
  default:
    timeoutSec: 600
    connectionDraining:
      drainingTimeoutSec: 300
    sessionAffinity:
      type: GENERATED_COOKIE
      cookieTtlSec: 3600
  targetRef:
    group: ""
    kind: Service
    name: gemini-session-svc
```

Notice three critical settings in this configuration:

`maxUnavailable: 0`` Ready` state before terminating any existing Pod holding active sessions.`sleep 10`)` timeoutSec: 600` and `drainingTimeoutSec: 300`
Even with a 300-second drain window, an agent session might span multiple user interactions over an hour, or a node preemption on Spot capacity might give only 30 seconds of warning. By checkpointing the conversation history after every Gemini 4 Argon turn into Redis (such as [Memorystore for Redis](https://cloud.google.com/memorystore/docs/redis)), any replica in the Deployment can transparently resume the session if the original Pod terminates.

Here is the core session server (`session_server.py`) showing graceful `SIGTERM` tracking and per-turn Redis checkpointing with the `google-genai` SDK:

``` python
import asyncio
import json
import os
import signal
from contextlib import asynccontextmanager
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel
import redis.asyncio as redis
from google import genai
from google.genai import types

draining = False
active_turns = 0

class TurnRequest(BaseModel):
    user_message: str

def handle_sigterm(*_args):
    global draining
    draining = True

@asynccontextmanager
async def lifespan(_app: FastAPI):
    signal.signal(signal.SIGTERM, handle_sigterm)
    yield
    # Wait for active Gemini 4 Argon turns to finish and checkpoint before exiting
    while active_turns > 0:
        await asyncio.sleep(1)
    await redis_client.aclose()

app = FastAPI(lifespan=lifespan)
genai_client = genai.Client(
    vertexai=True,
    project=os.environ["GOOGLE_CLOUD_PROJECT"],
    location=os.environ.get("VERTEX_LOCATION", "us-central1"),
)
redis_client = redis.Redis(host=os.environ["REDIS_HOST"], port=6379, decode_responses=True)
MODEL_ID = os.environ.get("GEMINI_MODEL", "gemini-4-argon")

@app.get("/healthz/ready")
async def readiness_check():
    if draining:
        raise HTTPException(status_code=503, detail="Pod is draining active sessions")
    return {"status": "ready"}

@app.post("/v1/sessions/{session_id}/stream")
async def stream_agent_turn(session_id: str, request: TurnRequest):
    # Load prior conversation history from Redis checkpoint
    raw_history = await redis_client.get(f"session:{session_id}")
    history = json.loads(raw_history) if raw_history else []
    history.append({"role": "user", "parts": [{"text": request.user_message}]})

    contents = [
        types.Content(role=turn["role"], parts=[types.Part.from_text(text=p["text"]) for p in turn["parts"]])
        for turn in history
    ]

    async def event_generator():
        global active_turns
        active_turns += 1
        assembled_response = []
        try:
            async for chunk in await genai_client.aio.models.generate_content_stream(
                model=MODEL_ID,
                contents=contents,
            ):
                if chunk.text:
                    assembled_response.append(chunk.text)
                    yield f"data: {json.dumps({'text': chunk.text})}\n\n"

            # Checkpoint completed assistant turn back to Redis with a 24-hour TTL
            history.append({"role": "model", "parts": [{"text": "".join(assembled_response)}]})
            await redis_client.set(f"session:{session_id}", json.dumps(history), ex=86400)
        finally:
            active_turns -= 1

    return StreamingResponse(event_generator(), media_type="text/event-stream")
```

Once deployed, confirm that VPA CPU startup boost is accelerating Pod initialization and resizing CPU in place without restarting the container:

```
# Inspect the startup boost annotation injected on the Pod
kubectl get pods -n agent-runtime \
    -l app=gemini-session-orchestrator \
    -o custom-columns="NAME:.metadata.name,CPU_REQ:.spec.containers[0].resources.requests.cpu,BOOST_ANNOTATION:.metadata.annotations.vpaCpuStartupBoost\/agent-server"

# Watch for the InPlaceResizedByVPA event once the Pod passes readinessProbe
kubectl get events -n agent-runtime --field-selector reason=InPlaceResizedByVPA
```

Expected event output:

```
LAST SEEN   TYPE     REASON                 OBJECT                                             MESSAGE
14s         Normal   InPlaceResizedByVPA    pod/gemini-session-orchestrator-6b8f9c7d4-k29lm    Pod was in-place resized by VPA Updater (CPU: 1500m -> 500m)
```

To verify that rolling updates drain cleanly without dropping streams, trigger a rollout while streaming a multi-step Gemini 4 Argon response and monitor active backend connections in [Cloud Monitoring](https://cloud.google.com/monitoring/docs) using the metric `loadbalancing.googleapis.com/https/backend_request_count` alongside `kubernetes.io/container/cpu/core_usage_time`.

`memory` requests and limits to accommodate the largest concurrent payload your agent processes.`spot: true` in a GKE Running Gemini 4 Argon agents reliably in production requires adapting Kubernetes from short-lived request/response defaults to stateful, long-horizon execution. By combining VPA CPU Startup Boost for fast initialization, `GCPBackendPolicy` for long streaming timeouts, and per-turn state checkpointing, your agent deployments can scale and roll out new versions without interrupting active users.

To dive deeper into these GKE capabilities, read the [VPA CPU Startup Boost guide](https://cloud.google.com/kubernetes-engine/docs/how-to/boost-application-startup), review the [GKE Gateway `GCPBackendPolicy` reference](https://cloud.google.com/kubernetes-engine/docs/how-to/configure-gateway-resources#configure_backend_service), and check the [Gemini API documentation on Google Cloud](https://cloud.google.com/vertex-ai/generative-ai/docs).