cd /news/ai-infrastructure/aws-lambda-now-supports-gpu-instance… · home › topics › ai-infrastructure › article
[ARTICLE · art-172081] src=aws.amazon.com ↗ pub= topic=ai-infrastructure verified=true sentiment=↑ positive

AWS Lambda now supports GPU instances for Event Functions on Lambda Managed Instances

Amazon Web Services added GPU support for Event Functions on AWS Lambda Managed Instances, letting customers run GPU-accelerated workloads on Amazon EC2 G-series GPU instances through serverless functions deployed as Open Container Initiative (OCI) images. AWS said Lambda handles provisioning, scaling and event integration with no GPU drivers, CUDA versions or AMIs to reconcile, and that GPU-accelerated functions support the extended function timeout on Lambda Managed Instances of up to 90 minutes for asynchronous and event source mapping invocations. AWS positions the capability for inference rather than training, including custom model inference, agentic event-driven AI, media transcoding, 3D rendering and scientific computing.

key takeaway

GPU-accelerated Lambda Event Functions run on EC2 G-series instances from OCI container images with no driver or CUDA management, and support timeouts up to 90 minutes for asynchronous and event source mapping invocations.

by read9 min views11 publishedNov 18, 2026
AWS Lambda now supports GPU instances for Event Functions on Lambda Managed Instances
Image: AWS ML Blog

AWS News Blog #

| |

GPU support is now available for EC2 G-series GPU instances for Event Functions running on AWS Lambda Managed Instances (LMI). You can now run GPU-accelerated workloads, including machine learning inference, media processing, rendering, and scientific computing, on GPU instances with serverless simplicity. GPU-accelerated functions are deployed as container (Open Container Initiative, OCI) images. You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration. There are no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate. GPU-accelerated functions run on the same Lambda Managed Instances foundation that already gives you Amazon EC2 pricing and specialized compute without managing servers.

Why customers need this

     Customers keep asking to run GPU workloads with the simplicity of serverless. Building on self-managed GPU infrastructure means selecting instances, installing and reconciling GPU drivers and CUDA libraries, planning capacity, writing scaling logic, and hand-building the event integrations that connect the GPU step to the rest of the application. Serverless GPU services that hide that work often sit outside your AWS environment, with no native event sources, identity, or networking.

Cost is the second problem. GPUs are expensive, and utilization is usually low because a single workload rarely saturates a GPU. A small inference task can leave most of a provisioned GPU idle while you pay for the whole thing.

This feature closes both gaps. You bring a function, and Lambda runs it on a GPU in minutes, triggered by the same AWS events as any other Lambda Event Function, billed and secured the same way. It also maximizes utilization of every GPU you pay for, by consolidating functions onto instances and keeping GPUs busy across concurrent requests.

What you can build

     You can run both AI/ML inference and non-ML workloads using GPU-accelerated compute on Lambda Managed Instances. This capability is optimized for inference rather than training, and custom models rather than large foundation models already available on services like [Amazon Bedrock](https://aws.amazon.com/bedrock/). Common workloads include:
  • Custom model inference : decision models (such asStrands Decider ), computer vision, speech-to-text and text-to-speech, recommendation, document analysis, and small-to-mid-size language models deployed as part of an application.
  • Agentic and event-driven AI : custom inference triggered directly by AWS events, composed with the rest of the Lambda programming model.
  • Media processing and transcoding : video and image pipelines triggered on upload.
  • 3D rendering and generative media : frame and asset rendering, diffusion, and text-to-video generation.
  • Scientific computing and financial modeling : simulations, matrix computation, and risk analysis.

Part of the Lambda programming model

     A GPU-accelerated function is one step in a larger application, and it composes with the rest of Lambda. [Lambda durable functions](https://aws.amazon.com/lambda/lambda-durable-functions/) orchestrate multi-step, long-running workflows around the GPU step, with checkpointing and automatic recovery. [Lambda MicroVMs](https://aws.amazon.com/lambda/lambda-microvms/) run untrusted or model-generated code in isolated, stateful sandboxes. In an agentic workload, a MicroVM executes the tool-use logic on CPU, then calls a GPU-accelerated function running on Lambda Managed Instances for inference.

GPU-accelerated functions also support the extended function timeout on Lambda Managed Instances, up to 90 minutes for asynchronous and event source mapping invocations, so you can run batch and other long-running GPU jobs, such as media transcoding or multi-step inference, without re-architecting. GPU-accelerated functions support container images up to 32 GB (10 GB per file), up to 32 GB of ephemeral storage, 2,048 to 32,768 MB of memory, 3 to 48 GB of accelerator memory, up to 48 concurrent requests per execution environment, and up to 15,000 vCPUs per capacity provider.

Let’s try it out

     To show what this looks like in practice, I deployed an open-source large language model (LLM) from Hugging Face as a Lambda function and sent it a prompt. With the vLLM base image from Lambda, I only set the Hugging Face model ID in the `MODEL_ID` environment variable, without writing any inference code. I used Qwen/Qwen2.5-1.5B-Instruct, a small instruction-tuned model that suits in-application tasks such as summarization, classification, and extraction. The flow has four steps: create a capacity provider, build and push the container image, create a function from that image, then publish and invoke. Before you begin, request an increase to your G-series instance quota (listed as *Running On-Demand G and VT instances*), because new accounts might start with a quota of 0. For more information, see [On-Demand Instance quotas](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-on-demand-instances.html#ec2-on-demand-instances-limits).

First, I created a capacity provider, where I specify the GPU instance types Lambda can use and let it auto-scale. Listing several instance types lets Lambda pick the most cost-effective one that is available and fall back across generations when capacity is tight.

{
    "CapacityProviderName": "lmi-gpu-cp",
    "VpcConfig": {
        "SubnetIds": ["subnet-aaaa", "subnet-bbbb", "subnet-cccc"],
        "SecurityGroupIds": ["sg-aaaa"]
    },
    "PermissionsConfig": {
        "CapacityProviderOperatorRoleArn": "arn:aws:iam::111122223333:role/lmi-operator-role"
    },
    "InstanceRequirements": {
        "AllowedInstanceTypes": [
            "g5.2xlarge", "g6.2xlarge", "g6e.2xlarge"
        ]
    },
    "CapacityProviderScalingConfig": {
        "ScalingMode": "Auto",
        "MaxVCpuCount": 96
    }
}
aws --region us-east-2 \
  lambda create-capacity-provider \
  --cli-input-json file://cli-skeletons/create-capacity-provider_gpu.json

The Lambda console shows the new capacity provider, lmi-gpu-cp, in the Active state. Note the VPC configuration with its subnets and security group.

Figure 1: The capacity provider is Active, with its VPC configuration.

Next, I built the container image. Lambda provides a curated vLLM base image that includes the vLLM inference server, PyTorch, Transformers, and the CUDA toolkit, along with a runtime interface client that supports concurrent invocations. The image serves the model for me, so I don’t write a handler or any inference server code. My entire Dockerfile is one line:

FROM public.ecr.aws/lambda/deep-learning-containers/vllm:cuda-v1

After pushing the image to Amazon ECR, I created the function. I chose the model with the MODEL_ID environment variable, which accepts a Hugging Face or ModelScope repository, an Amazon S3 location, or a local path. Lambda downloads the model, manages GPU memory, and serves requests. Qwen2.5-1.5B-Instruct is a public model, so I didn’t need a Hugging Face token. For private or gated models, or to get higher download rate limits, you can also set HF_TOKEN.

GPU-accelerated functions use AcceleratorConfig to request accelerator memory. AcceleratorMemorySize is the minimum GPU memory the function needs, and Lambda places the execution environment on the smallest available GPU that meets it, so the GPU you get can have more memory than you requested. I set PerExecutionEnvironmentMaxConcurrency to 4, the default. The vLLM server batches concurrent requests on the GPU, so invocations share one copy of the model and the GPU stays busy. I also set ExecutionEnvironmentMemoryGiBPerVCpu to 4, which gives the function the 16,384 MB of memory and 4 vCPUs it needs while leaving capacity on the 2xlarge instance for Lambda’s management components.

{
    "FunctionName": "lmi-gpu-inference",
    "PackageType": "Image",
    "Code": { "ImageUri": "111122223333.dkr.ecr.us-east-2.amazonaws.com/lmi-gpu-inference:latest" },
    "Role": "arn:aws:iam::111122223333:role/lmi-gpu-inference-execution-role",
    "Timeout": 900,
    "MemorySize": 16384,
    "EphemeralStorage": { "Size": 32768 },
    "Environment": { "Variables": { "MODEL_ID": "Qwen/Qwen2.5-1.5B-Instruct" } },
    "CapacityProviderConfig": {
        "LambdaManagedInstancesCapacityProviderConfig": {
            "CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp",
            "ExecutionEnvironmentMemoryGiBPerVCpu": 4,
            "PerExecutionEnvironmentMaxConcurrency": 4
        }
    },
    "AcceleratorConfig": { "AcceleratorMemorySize": 16 }
}
aws --region us-east-2 \
  lambda create-function \
  --cli-input-json file://cli-skeletons/create-gpu-llm-function.json

The function overview shows what makes this a GPU-accelerated function: the attached capacity provider that supplies the GPU instances and the vLLM container image deployed from Amazon ECR, all wired into the same function model you already use.

Figure 2: The lmi-gpu-inference function overview, showing the attached capacity provider and the vLLM container image deployed from Amazon ECR.

Functions on Lambda Managed Instances run from a published version, so I published one. Publishing provisions the execution environments, and each one downloads the model weights and warms up the engine before the version becomes active. The cold start happens during the publish step, so the first caller doesn’t wait for it.

aws --region us-east-2 \
  lambda publish-version \
  --function-name lmi-gpu-inference \
  --publish-to LATEST_PUBLISHED

Then I sent the model a prompt. The payload is a standard OpenAI request body: a messages array routes to chat completions, and a prompt field routes to text completions.

aws --region us-east-2 \
  lambda invoke \
  --function-name 'lmi-gpu-inference:$LATEST.PUBLISHED' \
  --cli-binary-format raw-in-base64-out \
  --payload '{"messages": [{"role": "user", "content": "Explain serverless GPU inference in one sentence."}], "max_tokens": 128}' \
  /dev/stdout

The model answered from a GPU inside the Lambda execution environment, and the response is a standard OpenAI completion object, so any OpenAI-compatible client works with it.

{
  "model": "Qwen/Qwen2.5-1.5B-Instruct",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Serverless GPU inference is a cloud-based service that allows for efficient execution of machine learning models on NVIDIA GPUs without the need for infrastructure management or maintenance."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 39, "completion_tokens": 31, "total_tokens": 70 }
}

Four concurrent invocations finished in about the same time as a single one, on one execution environment that shares one copy of the model in GPU memory. From a Hugging Face model ID to a live, OpenAI-compatible GPU endpoint took about 10 minutes, most of it unattended while Lambda provisioned capacity and warmed up the model.

That’s a working LLM endpoint with no inference server code, no GPU drivers, and no cluster to manage. To try a different model, I update MODEL_ID and publish a new version, with no image rebuild. Each version is an immutable snapshot, so different models can run side by side as separate versions. It is also a Lambda Event Function, so I can trigger it from an Amazon SQS queue to process documents in batches, put Amazon API Gateway in front of it, or call it as the inference step in a Lambda durable functions workflow.

For production, copy the model weights to an S3 bucket in the same Region, point MODEL_ID at that bucket, and set VLLM_LOAD_FORMAT=runai_streamer so the weights stream straight into memory. This gives faster starts and removes the dependency on the Hugging Face Hub. For more guidance, see the best practices for running inference workloads. Beyond LLM inference, you can also run custom ML inference, media processing, and other GPU workloads. See the Lambda documentation for best practices.

The configuration tab shows the settings that matter for a GPU-accelerated function: 16,384 MB of memory, the 15-minute timeout, and 32 GB of ephemeral storage for the downloaded weights, alongside the function-level capacity provider settings: per-environment concurrency of 4, the 4:1 memory-to-vCPU ratio, and 16 GB of accelerator memory.

Figure 3: The function’s configuration, including memory, timeout, and 32 GB of ephemeral storage, and the function-level capacity provider settings: per-environment concurrency, the memory-to-vCPU ratio, and accelerator memory.

When you finish testing, delete the function and then the capacity provider so that Lambda stops the GPU instances and you stop paying for them.

How it works

     The capabilities below are what you get out of the box, each one removing GPU work you would otherwise own.

Skip the GPU software setup. Curated base images ship with the framewo

── more in #ai-infrastructure 4 stories · sorted by recency
── more on @amazon web services 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/aws-lambda-now-suppo…] indexed:0 read:9min 2026-11-18 · —