AWS Lambda Now Runs GPU Inference with 48 Concurrent Invocations Sharing One Model per GPU AWS added GPU support to AWS Lambda, letting Event Functions on Lambda Managed Instances run on EC2 G-series instances (G4dn, G5, G6, G6f, Gr6, Gr6f and G6e at launch) deployed as container images, with up to 48 concurrent invocations per environment sharing a single copy of the model in GPU memory. The feature is generally available in commercial regions where both Managed Instances and G-series instances exist and is priced at the EC2 On-Demand rate plus a 9.75 percent management fee and $0.20 per million requests. AWS senior specialist solutions architect Daniel Abib said operators build an OCI image and point the function at it, with "no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate. AWS has added GPU support https://aws.amazon.com/blogs/aws/aws-lambda-now-supports-gpu-instances-for-event-functions-on-lambda-managed-instances/ to AWS Lambda: Event Functions running on Lambda Managed Instances https://aws.amazon.com/about-aws/whats-new/2026/11/aws-lambda-gpu-managed-instances/ can now run on EC2 G-series instances G4dn, G5, G6, G6f, Gr6, Gr6f and G6e at launch , deployed as container images, and triggered by the same event sources as any other function. The feature is generally available in the commercial regions where both Managed Instances and G-series instances exist. It is priced at the EC2 On-Demand rate plus a 9.75 percent management fee and the standard $0.20 per million requests. As InfoQ reported https://www.infoq.com/news/2025/12/aws-lambda-managed-instances last December, Managed Instances let a function run on EC2 capacity the customer selects and pays for, with Lambda handling provisioning, patching and scaling and several invocations sharing one execution environment. GPUs were missing at that launch. Daniel Abib, senior specialist solutions architect at AWS, describes the operator's side of the change: You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration. There are no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate. A capacity provider lists the allowed instance types, the VPC and a vCPU ceiling; Lambda picks the newest generation with capacity and falls back to older generations when newer ones are short: "InstanceRequirements": { "AllowedInstanceTypes": "g5.2xlarge", "g6.2xlarge", "g6e.2xlarge" }, "CapacityProviderScalingConfig": { "ScalingMode": "Auto", "MaxVCpuCount": 96 } The function declares its GPU memory through a new AcceleratorConfig block, next to memory, timeout, storage, and a per-environment concurrency: "MemorySize": 16384, "Timeout": 900, "EphemeralStorage": { "Size": 32768 }, "Environment": { "Variables": { "MODEL ID": "Qwen/Qwen2.5-1.5B-Instruct" } }, "CapacityProviderConfig": { "LambdaManagedInstancesCapacityProviderConfig": { "CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp", "ExecutionEnvironmentMemoryGiBPerVCpu": 4, "PerExecutionEnvironmentMaxConcurrency": 4 } }, "AcceleratorConfig": { "AcceleratorMemorySize": 16 } Concurrent invocations in one environment share a single copy of the model in GPU memory, up to 48 per environment, and scaling follows GPU utilization, CPU utilization, and concurrency. Publishing a version provisions the environments, downloads the weights, and warms the engine before the first event arrives; Seekable OCI https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-gpu-startup.html lazy loading can cut environment start time by up to 70 percent. AWS ships five GPU base images https://docs.aws.amazon.com/lambda/latest/dg/lambda-managed-instances-gpu-base-images.html : CUDA only, CuPy, PyTorch, vLLM, and SGLang. The two inference servers need no handler code; a MODEL ID variable points to a Hugging Face or ModelScope repository, an S3 prefix, or a local path, and a model swap is a configuration change followed by a new version: aws lambda update-function-configuration \ --function-name my-gpu-function \ --environment 'Variables={MODEL ID=s3://amzn-s3-demo-bucket/models/Qwen2.5-1.5B-Instruct/,VLLM LOAD FORMAT=runai streamer}' The limits are in the launch post: container images up to 32 GB, 32 GB of ephemeral storage, 2 to 32 GB of memory, 3 to 48 GB of accelerator memory, 15,000 vCPUs per capacity provider, and a timeout of up to 90 minutes for asynchronous and event source mapping invocations. Abib draws the line against the competition in one sentence: serverless GPU services that hide the same work "often sit outside your AWS environment, with no native event sources, identity, or networking." The first hands-on reports came within a day. Classmethod's Akaike, who had corrected a conference talk in December after finding that no G-type was accepted, tried every accelerated instance type https://dev.classmethod.jp/en/articles/lambda-managed-instances-gpu-acceleratorconfig/ in us-east-1 and found 37 of 81 accepted, all G-family; P-family, G7, Inferentia, Trainium, and the Graviton-based g5g were refused, as were zip packages. A PyTorch benchmark ran about 66 times faster on the L4 than on the instance's CPU. Still, neither Terraform nor the Cloud Control provider knew AcceleratorConfig yet, and the GPU vCPU quota in a fresh account is zero: "if you use g6.xlarge 4 vCPU with the default three instances, you need a quota of 12 vCPUs." A Terraform provider issue https://github.com/hashicorp/terraform-provider-aws/issues/51046 and pull request, opened on 16 November by a user named sasidhar-aws, and a pull request from the same user remain open. A second test, by an AWS Community Builder writing as xbill, put Gemma 4 E2B https://dev.to/aws-builders/gemma-4-on-aws-lambda-with-a-gpu-e2b-on-an-nvidia-l4-day-one-4ej on a single g6.2xlarge through the vLLM image, with Google's 4-bit QAT weights and a speculative-decoding drafter baked into a 9 GB image. One environment served all 48 simultaneous requests and every one of 417 calls succeeded, at roughly 130 tokens a second for a single request and 3,400 to 3,600 at 48. The friction was in the pipeline rather than the GPU: 484 seconds from publish to an active version, a Docker image index that Lambda rejected until it was pushed as a single-platform manifest, and a handler that returns complete replies, so time to first token could not be measured. His verdict keeps to that scope: For a model behind events, queues or other AWS services, the Lambda GPU function is the simpler road: one image, one function, and Lambda keeps the server alive and routes the requests. Keep the drafter on for a chat with a few users at a time, and publish without it for heavy batch load. When streamed replies or full control of the server are needed, he writes, "run vLLM on an EC2 G6 yourself." The pricing model is the one Managed Instances introduced, with a lower fee for GPUs than the 15 percent charged on CPU instance types. Savings Plans and Reserved Instances apply to the instance, and there is no duration charge, so an instance bills for as long as the minimum environment count is above zero. Yan Cui's December analysis https://theburningmonk.com/2025/12/what-you-need-to-know-about-lambda-managed-instances/ of the model, written when it still carried the line "GPU instances are NOT supported," applies to the GPU case unchanged: Regular Lambda functions can scale rapidly by tapping into a large, shared pool of EC2 instances. With managed instances, it takes tens of seconds to launch new EC2 instances. As such, it's not a good fit when you have large, unpredictable spikes in traffic. Managed Instances still support only Event Functions, so the GPU path does not yet reach Web Functions, the HTTP-server shape Lambda gained this year, and AWS has not said whether it will.