31.8x Speedup by Changing One File: Async Embedding Calls on AWS Bedrock

A developer achieved a 31.8x speedup in a RAG ingestion pipeline by converting sequential HTTP embedding calls to asynchronous concurrent requests using aiohttp and asyncio.gather on AWS Bedrock, reducing processing time from 49.61 seconds to 1.56 seconds with no infrastructure changes.

I was testing a RAG ingestion pipeline and noticed something painful: ingesting a single document took nearly 50 seconds. Checking the metrics revealed that the CPU was doing almost nothing. The app wasn't compute-bound — it was simply sitting idle, waiting for sequential HTTP network round-trips to return. By converting the blocking embedding module into an asynchronous workflow, processing time dropped from 49.61 seconds to 1.56 seconds with zero infrastructure changes. us-east-1 | Approach | Time | Difference | |---|---|---| Sequential requests blocking loop | 49.61s | Baseline | Concurrent aiohttp + asyncio.gather | 1.56s | 31.8× faster | Each request must complete before the next one starts. chunk 1 ── request ── ⏳ wait ~1.5s ── response chunk 2 ── request ── ⏳ wait ~1.5s ── response chunk 3 ── request ── ⏳ wait ~1.5s ── response ... Total Time ≈ 33 chunks × ~1.5s ≈ 49.6s Since network latency dominates the runtime, the event loop sits completely unused. Requests are dispatched concurrently; total time collapses down to the duration of the slowest single request. chunk 1 ── request ────────────── response chunk 2 ── request ────────────── response chunk 3 ── request ────────────── response ... in flight concurrently Total Time ≈ slowest single request ≈ 1.56s php import requests def generate embeddings texts: list str - list list float : all embeddings = for text in texts: response = requests.post url, headers=headers, json={"inputText": text} vector = response.json .get "embedding", all embeddings.append vector return all embeddings python import asyncio import aiohttp async def fetch embedding session: aiohttp.ClientSession, text: str - list float : async with session.post url, headers=headers, json={"inputText": text} as res: data = await res.json return data.get "embedding", async def generate embeddings async texts: list str - list list float : async with aiohttp.ClientSession as session: tasks = fetch embedding session, text for text in texts return await asyncio.gather tasks Note: If you're using boto3 instead of raw HTTP requests, look at aioboto3, or offload the blocking SDK calls with asyncio.to thread to get similar non-blocking behaviour. | Chunks | Sequential est. | Concurrent est. | |---|---|---| | 31 | 49.61s | 1.56s | | 100 | ~160s 2.7 min | ~2–3s | | 500 | ~800s 13 min | ~3–5s | | 1,000 | ~1,600s 26 min | ~5–10s | The more chunks you ingest, the more the concurrent approach pays off — batch ingestion jobs that took half an hour now finish in seconds. asyncio.Semaphore before you hit them. embedder.py removed the embedding stage as a bottleneck in the pipeline.