Defeating the Fargate Cold Start Chaos with SOCI

A developer tested AWS Seekable OCI (SOCI) on Fargate to reduce cold start times. SOCI cut median cold start from 49s to 35s (29% faster) and reduced variability, but showed no benefit on warm starts. The engineer found that ECR caching skewed results and that building the SOCI index builder from source was necessary due to missing prebuilt artifacts.

If you've ever hit deploy on AWS Fargate and watched the task sit in PENDING forever, this one's for you. I ran a small experiment with Seekable OCI SOCI the AWS thing that promises to fix Fargate cold starts. This article is what I learned. The good, the boring, and the "wait, that didn't work?" bits. You'll walk away knowing what SOCI https://docs.aws.amazon.com/sagemaker/latest/dg/soci-indexing.html is, whether you should bother setting it up, and what real numbers look like That's it. Fargate is great. Push a container, get a running task. No nodes to manage Until you use a big image. Every Fargate task pulls the whole image before starting. No shared cache. No head start. If your image is 3 GB, your task waits for 3 GB to download. Every time. Every task. For an ML model or a heavy Java service, cold start becomes minutes, not seconds. Two stats made me actually care about this: So you're pulling 100% to use 6.4%. And it costs you 76% of your startup budget doing it. That's ridiculous This isn't a Fargate-only problem, by the way. It's a container problem. But Fargate hurts more because you can't cache anything at the host level. I presented this talk at AWS Community Days Bangalore 2026 https://acd.awsugblr.in/ SOCI stands for Seekable OCI . Simple idea: Instead of downloading the whole image before starting the container, download only the parts the app needs right now. Everything else gets pulled lazily, in the background, while the app is already running. The way it works: No code changes. No Dockerfile changes. Same image. Just an extra index file sitting next to it in ECR. That's it. That's the whole trick. I wanted to see if this actually works. Not "does it work in AWS's demo videos" does it work on a real image I might ship. Three acts. Same test, different pressure. Act I — Baseline vs SOCI, one task at a time. Three cold starts each. No load, no chaos. Just: how fast does one task come up? Act II — Thundering herd. Launch 10 tasks of each variant at the same time. This stresses ECR bandwidth. Real production scale-out looks like this. Act III — Scale-out under existing load. Start a small fleet, then trigger a scale-out on top. Simulates adding capacity when you already have tasks running. Between each act I stopped the tasks and waited. The setup logged every task's RunTask → RUNNING → responsive timestamps to a CSV. Then a small script aggregated everything into an HTML report. Prove SOCI is faster. Under normal load and under stress. The SOCI Index Builder the AWS Labs thing that auto-generates the index is published as source only. There's no public S3 bucket with prebuilt Lambda zips you have to build them yourself. I hit an AccessDenied error trying to sync from a bucket that doesn't exist. Wasted an hour on that. If you set this up: build the Lambdas from the source repo, host them in your own bucket. Here's the honest data from my report. Median cold start: 49s → 35s. About 29% faster . Not the 10× win the marketing decks promise, but real. The interesting number is the standard deviation. Baseline swings by 13 seconds run-to-run. SOCI swings by 5. SOCI is more predictable . That matters more than raw speed if you're sizing auto-scaling. Wait the baseline dropped from 49s to 29s under concurrent load? That doesn't make sense... unless something got cached. Yeah. ECR caches. By the time Act II ran, the image was warm in ECR. So the "cold" start in Act II wasn't actually cold. Both variants got the benefit. Lesson: benchmarking cold starts is harder than it looks. Warm caches make results lie. SOCI is slightly slower here. 😅 Not a bug. With everything warm, the benefit of skipping bytes evaporates. And SOCI has its own small overhead — fetching the index, doing lazy reads. On a fully warm system, that overhead can cost you. Takeaway: SOCI wins on genuinely cold starts. On warm systems, don't expect miracles. Short answer: yes, if your image is over 1 GB. Below 250 MB, don't bother. The overhead isn't worth it. Above 1 GB, especially for ML models or heavy JVM/Python services, this is exactly what SOCI was built for. SOCI is free. You only pay for storing the index in ECR, which is tiny. So there's not much reason not to try it. The setup is a bit annoying you have to build the Index Builder Lambdas from source and host them yourself. But once it's running, it's automatic. Push an image, get an index. Done. If you liked this content, follow me on Twitter at kitarp29 https://twitter.com/kitarp29 for more Thanks for reading my article :