# Benchmarking Serverless GPUs: Modal vs RunPod vs Replicate Cold Starts (2026)

> Source: <https://dev.to/mrzitoun/benchmarking-serverless-gpus-modal-vs-runpod-vs-replicate-cold-starts-2026-a5c>
> Published: 2026-09-03 20:13:13+00:00

Deploying open-source LLMs (like Llama-3) or real-time Whisper transcription in production often forces a difficult architectural trade-off: keep dedicated GPUs running 24/7 (expensive) or rely on serverless scale-to-zero (cold start latency penalty).

To evaluate container spin-up overhead, we benchmarked median cold start latencies and per-second execution costs across the major serverless GPU platforms.

| Provider | GPU | Median Cold Start | Equiv. Hourly Rate | Scale-To-Zero |
|---|---|---|---|---|
Modal |
A100 (40GB) | 1.8s | ~$2.85 / hr | Yes |
RunPod Serverless |
A100 (80GB) | 4.2s | ~$2.59 / hr | Yes |
Replicate |
A100 (80GB) | 6.5s | ~$4.14 / hr | Yes |
Together AI |
H100 Cluster | Instant (Pooled) | Token-based | N/A |
Lambda Labs |
A100 (80GB) | VM Boot (~45s) | $1.89 / hr | No |

The full benchmark dataset, hardware configurations, and testing scripts are maintained at [ServerlessGPUBench](https://serverlessgpubench.com).

Raw benchmark metrics are also open-sourced on GitHub: [awesome-serverless-gpu-latency](https://github.com/mrzitoun/awesome-serverless-gpu-latency).