# GPUs For Actors In Agent Substrate

> Source: <https://dev.to/thenjdevopsguy/gpus-for-actors-in-agent-substrate-11p7>
> Published: 2026-09-01 13:18:32+00:00

The increase in cost of GPUs and AI is hitting an all-time high, so much so that it's hard to even get GPU usage approved within a public cloud environment. Because of that, the ability to splice/share GPUs across workloads that need GPUs (apps, Agents) is a necessity not only for cost savings, but for hardware/resource savings as GPUs aren't an unlimited resource (especially nowadays).

In this blog post, you'll learn how to implement CUDA (NVIDIA's parallel processing) inside an Agent Substrate Actor and see how many intermittent GPU workloads can take turns using a smaller pool of GPU-backed Substrate Workers.

💡How Agents work today (tool calls, planning, loops, etc.) are CPU-centric tasks. That means the majority of the time, you won't see an Agent that needs a GPU. However, there are a few edge cases (neural-network math that's local, the LLM is in-process where the GPU is for the LLM, but it lives inside the Agent process, a tool the Agent runs is a GPU job like local image/vid gen) where an Agent using a GPU makes sense. This blog, however, uses a standard HTTP app inside of the Actor.

To follow along with this blog post from a hands-on perspective, you will need:

`PodCertificate`

API needs the ability to be enabled on the k8s API server.`KO_DOCKER_REPO`

and `BUCKET_NAME`

from your Substrate env file (found in the repo from step 3 that you can edit).`substrate/optimization/gpu-backed-actors`

to see what you're deploying and you'll need it to build the images later.GPU splicing isn't new. With the NVIDIA Operator, we've been able to do that within Kubernetes for quite a while.

Example:

```
apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
data:
  any: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        renameByDefault: false
        failRequestsGreaterThanOne: false
        resources:
          - name: nvidia.com/gpu
            replicas: 4
```

The above shows that the NVIDIA GPU is deployed on a Worker Node within your k8s cluster and up to four (4) Pods can request a "piece" of the one GPU.

You'd do that by putting a parameter like this within your Pods Manifest:

```
nvidia.com/gpu: 1
```

Instead of splicing per Pod, you can splice per Actor (an Agent or HTTP app runs inside an Actor), which saves hardware resources and therefore cost because multiple Actors can run in one Worker Pod.

💡As it stands right now, one Actor can run at a time within a Worker, which means the GPU is being used by said Actor.

In the next section, you'll see how to ensure an Actor can use a NVIDIA GPU.

A Substrate Actor can use a GPU the same way a Pod does, which is by putting `nvidia.com/gpu`

on the Worker/WorkerPool. There is no GPU field on `ActorTemplate`

. Instead, Substrate passes the assigned device into every container in the Actor.

```
source /path/to/substrate/.ate-dev-env.sh
export SUBSTRATE_DIR=/path/to/substrate

kubectl get pods -n ate-system
kubectl get sandboxconfig gvisor-default
kubectl get nodes -o json | jq -r \
  '["NAME","GPU","GKE_ACCEL","PRODUCT"],
   (.items[] | [
     .metadata.name,
     (.status.allocatable["nvidia.com/gpu"] // "-"),
     (.metadata.labels["cloud.google.com/gke-accelerator"] // "-"),
     (.metadata.labels["nvidia.com/gpu.product"] // "-")
   ]) | @tsv' | column -t
```

You'll see an output similar to the below:

In the next section, you'll implement an Actor with a usable NVIDIA GPU.

At the time of writing this, only gVisor (software-level isolation) based Actors can support GPUs.

```
CRD rule on WorkerPool:

nvidia.com/gpu is only supported when sandboxClass is 'gvisor'
```

The source of this is via the kubebuilder CEL marker on `WorkerPoolSpec`

.

Because of that, you will have to ensure that you have gVisor enabled on your cluster running Substrate.

💡The default `sandboxClass`

is gVisor, so if you didn't specify microVM during installation, you're good to go.

The images in this step are for the next step, but let's break down the "why" in terms of why we need them:

`gcr.io/distroless/static-debian13`

. It cannot exec nvidia-ctk, so it cannot inject a GPU into the sandbox.`workload/`

Which you can find here, is the Actors application (it's a Go app) that calls out to the GPU. It runs nvidia-smi inside the Actor as a child process.

```
export ATEOM_GPU_IMAGE=$(
  cd "$SUBSTRATE_DIR" &&
  KO_DOCKER_REPO="$KO_DOCKER_REPO" \
  KO_DEFAULTPLATFORMS=linux/amd64 \
  KO_DEFAULTBASEIMAGE=debian:stable-slim \
  ./hack/run-tool.sh ko build ./cmd/ateom-gvisor
)
echo "$ATEOM_GPU_IMAGE"
```

`ActorTemplate`

can use, which is used as a golden image to deploy an Actor with GPU needs.

```
cd agentic-demo-repo/substrate/optimization/gpu-backed-actors/workload

docker buildx build \
  --platform linux/amd64 \
  --push \
  --provenance=false \
  --metadata-file /tmp/gpu-agent.json \
  --tag "${KO_DOCKER_REPO}/gpu-actor-workload:gpu-agent" \
  workload/

export GPU_WORKLOAD_IMAGE="${KO_DOCKER_REPO}/gpu-actor-workload@$(jq -er '."containerimage.digest"' /tmp/gpu-agent.json)"
echo "$GPU_WORKLOAD_IMAGE"
```

`ActorTemplate`

.

```
export SNAPSHOT_LOCATION="gs://${BUCKET_NAME}/ate-demo-gpu/"
```

With the proper images built for both the `WorkerPool`

to have the ability to inject GPUs and the GPU-based image so the Actor uses an image that requires a GPU for the workload to run, let's deploy the resources.

`WorkerPool`

`nvidia.com/gpu: "1"`

in requests and limits is what injects the GPU into the sandbox.```

kubectl apply -f - <<EOF

apiVersion: v1

kind: Namespace

metadata:

name: ate-demo-gpu

labels:

apiVersion: ate.dev/v1alpha1

kind: WorkerPool

metadata:

name: gpu-workers

namespace: ate-demo-gpu

labels:

workload: gpu

spec:

replicas: 1

sandboxClass: gvisor

workerImage: ${ATEOM_GPU_IMAGE}

template:

tolerations:

- key: nvidia.com/gpu

operator: Exists

effect: NoSchedule

resources:

requests:

cpu: 500m

memory: 2Gi

nvidia.com/gpu: "1"

limits:

cpu: "2"

memory: 4Gi

nvidia.com/gpu: "1"

EOF

`

`ActorTemplate`

, which is the golden image/template that an Actor uses as a blueprint when it's created. Notice that it's using the GPU workload image.💡There is no GPU field on the template. The GPU comes from the `WorkerPool`

.

```

kubectl apply -f - <<EOF

apiVersion: ate.dev/v1alpha1

kind: ActorTemplate

metadata:

name: gpu-app

namespace: ate-demo-gpu

spec:

sandboxClass: gvisor

workerSelector:

matchLabels:

workload: gpu

containers:

With the `WorkerPool`

where the Actor runs and the Actor's golden image/template/blueprint created, you can now create the Actor.

kubectl ate create atespace gpu-demo

kubectl ate create actor gpu-1 --atespace gpu-demo --template ate-demo-gpu/gpu-agent

kubectl ate resume actor gpu-1 --atespace gpu-demo --boot

kubectl ate logs actors gpu-1 --atespace gpu-demo

You can test the Actor to ensure that it's working as expected:

```

kubectl -n ate-system port-forward svc/atenet-router 8000:80

curl -sS \

-H 'Host: gpu-1.gpu-demo.actors.resources.substrate.ate.dev' \

[http://localhost:8000/gpu](http://localhost:8000/gpu)

`

"No running processes found" is expected. `GET /gpu`

runs `nvidia-smi`

as a child and then exits. The Processes table only lists jobs that currently hold GPU memory (a CUDA context).