{"slug": "31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock", "title": "31.8x Speedup by Changing One File: Async Embedding Calls on AWS Bedrock", "summary": "A developer achieved a 31.8x speedup in a RAG ingestion pipeline by converting sequential HTTP embedding calls to asynchronous concurrent requests using aiohttp and asyncio.gather on AWS Bedrock, reducing processing time from 49.61 seconds to 1.56 seconds with no infrastructure changes.", "body_md": "I was testing a RAG ingestion pipeline and noticed something painful: ingesting a single document took nearly 50 seconds.\n\nChecking the metrics revealed that the CPU was doing almost nothing. The app wasn't compute-bound — it was simply sitting idle, waiting for sequential HTTP network round-trips to return.\n\nBy converting the blocking embedding module into an asynchronous workflow, processing time dropped from **49.61 seconds to 1.56 seconds** with zero infrastructure changes.\n\n`us-east-1`\n\n| Approach | Time | Difference |\n|---|---|---|\nSequential (`requests` blocking loop) |\n49.61s |\nBaseline |\nConcurrent (`aiohttp` + `asyncio.gather` ) |\n1.56s |\n31.8× faster |\n\nEach request must complete before the next one starts.\n\n```\nchunk 1 ──> request ──> ⏳ wait (~1.5s) ──> response\nchunk 2 ──> request ──> ⏳ wait (~1.5s) ──> response\nchunk 3 ──> request ──> ⏳ wait (~1.5s) ──> response\n...\nTotal Time ≈ (33 chunks × ~1.5s) ≈ 49.6s\n```\n\nSince network latency dominates the runtime, the event loop sits completely unused.\n\nRequests are dispatched concurrently; total time collapses down to the duration of the slowest single request.\n\n```\nchunk 1  ──> request ──────────────> response\nchunk 2  ──> request ──────────────> response\nchunk 3  ──> request ──────────────> response\n...        (in flight concurrently)\nTotal Time ≈ slowest single request ≈ 1.56s\nphp\nimport requests\n\ndef generate_embeddings(texts: list[str]) -> list[list[float]]:\n    all_embeddings = []\n    for text in texts:\n        response = requests.post(url, headers=headers, json={\"inputText\": text})\n        vector = response.json().get(\"embedding\", [])\n        all_embeddings.append(vector)\n    return all_embeddings\npython\nimport asyncio\nimport aiohttp\n\nasync def fetch_embedding(session: aiohttp.ClientSession, text: str) -> list[float]:\n    async with session.post(url, headers=headers, json={\"inputText\": text}) as res:\n        data = await res.json()\n        return data.get(\"embedding\", [])\n\nasync def generate_embeddings_async(texts: list[str]) -> list[list[float]]:\n    async with aiohttp.ClientSession() as session:\n        tasks = [fetch_embedding(session, text) for text in texts]\n        return await asyncio.gather(*tasks)\n```\n\n*Note: If you're using boto3 instead of raw HTTP requests, look at aioboto3, or offload the blocking SDK calls with asyncio.to_thread() to get similar non-blocking behaviour.*\n\n| Chunks | Sequential (est.) | Concurrent (est.) |\n|---|---|---|\n| 31 | 49.61s | 1.56s |\n| 100 | ~160s (2.7 min) | ~2–3s |\n| 500 | ~800s (13 min) | ~3–5s |\n| 1,000 | ~1,600s (26 min) | ~5–10s |\n\nThe more chunks you ingest, the more the concurrent approach pays off — batch ingestion jobs that took half an hour now finish in seconds.\n\n`asyncio.Semaphore`\n\n) before you hit them.`embedder.py`\n\n) removed the embedding stage as a bottleneck in the pipeline.", "url": "https://wpnews.pro/news/31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock", "canonical_source": "https://dev.to/edwardyun/318x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock-4l61", "published_at": "2026-07-27 07:16:14+00:00", "updated_at": "2026-07-27 07:33:07.680287+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "developer-tools", "ai-infrastructure"], "entities": ["AWS Bedrock", "aiohttp", "asyncio", "boto3", "aioboto3"], "alternates": {"html": "https://wpnews.pro/news/31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock", "markdown": "https://wpnews.pro/news/31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock.md", "text": "https://wpnews.pro/news/31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock.txt", "jsonld": "https://wpnews.pro/news/31-8x-speedup-by-changing-one-file-async-embedding-calls-on-aws-bedrock.jsonld"}}