31.8x Speedup by Changing One File: Async Embedding Calls on AWS Bedrock

I was testing a RAG ingestion pipeline and noticed something painful: ingesting a single document took nearly 50 seconds.

Checking the metrics revealed that the CPU was doing almost nothing. The app wasn't compute-bound — it was simply sitting idle, waiting for sequential HTTP network round-trips to return.

By converting the blocking embedding module into an asynchronous workflow, processing time dropped from 49.61 seconds to 1.56 seconds with zero infrastructure changes.

us-east-1

Each request must complete before the next one starts.

chunk 1 ──> request ──> ⏳ wait (~1.5s) ──> response
chunk 2 ──> request ──> ⏳ wait (~1.5s) ──> response
chunk 3 ──> request ──> ⏳ wait (~1.5s) ──> response
...
Total Time ≈ (33 chunks × ~1.5s) ≈ 49.6s

Since network latency dominates the runtime, the event loop sits completely unused.

Requests are dispatched concurrently; total time collapses down to the duration of the slowest single request.

chunk 1  ──> request ──────────────> response
chunk 2  ──> request ──────────────> response
chunk 3  ──> request ──────────────> response
...        (in flight concurrently)
Total Time ≈ slowest single request ≈ 1.56s
php
import requests

def generate_embeddings(texts: list[str]) -> list[list[float]]:
    all_embeddings = []
    for text in texts:
        response = requests.post(url, headers=headers, json={"inputText": text})
        vector = response.json().get("embedding", [])
        all_embeddings.append(vector)
    return all_embeddings
python
import asyncio
import aiohttp

async def fetch_embedding(session: aiohttp.ClientSession, text: str) -> list[float]:
    async with session.post(url, headers=headers, json={"inputText": text}) as res:
        data = await res.json()
        return data.get("embedding", [])

async def generate_embeddings_async(texts: list[str]) -> list[list[float]]:
    async with aiohttp.ClientSession() as session:
        tasks = [fetch_embedding(session, text) for text in texts]
        return await asyncio.gather(*tasks)

Note: If you're using boto3 instead of raw HTTP requests, look at aioboto3, or offload the blocking SDK calls with asyncio.to_thread() to get similar non-blocking behaviour.

The more chunks you ingest, the more the concurrent approach pays off — batch ingestion jobs that took half an hour now finish in seconds.

asyncio.Semaphore

) before you hit them.embedder.py

) removed the embedding stage as a bottleneck in the pipeline.