{"slug": "limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise", "title": "Limiting concurrent async calls in Node.js without a library: a 15-line promise pool", "summary": "A developer published a 15-line, dependency-free promise pool for Node.js that caps concurrent async calls at a fixed limit instead of firing every request at once via Promise.all. The mapLimit helper uses N lanes pulling from a shared index, preserves input order in results, and captures failures as values so one error doesn't abort the batch; tests confirm a peak of 4 in-flight calls versus 20 with Promise.all. The writeup advises limits of 4–16 for LLM APIs paired with a token-bucket rate limiter, and worker threads for CPU-bound work.", "body_md": "`Promise.all(items.map(callApi))` starts **every call at the same moment**. With 20 items that's fine. With 2,000 document pages to OCR, or 2,000 prompts to send to an LLM, it opens 2,000 connections, hits the provider's rate limit within a second, and can run the process out of sockets or memory. The fix is a **concurrency limit**: run at most N calls at a time and start the next as soon as one finishes. It takes about **15 lines and no dependency**: start N \"lanes\" that each pull the next item from a shared index until none are left. The version below also **keeps results in input order** and **records failures without stopping** the rest of the batch.\n\n``` js title=\"pool.mjs\"`worker\n\n/** Run`over` items`with at most` limit` in flight. Results keep input order; one failure doesn't stop the rest. */\n\nexport async function mapLimit(items, limit, worker) {\n\n  const results = new Array(items.length);\n\n  let next = 0;\n\n  async function lane() {\n\n    while (next < items.length) {\n\n      const i = next++;\n\n      try {\n\n        results[i] = { ok: true, value: await worker(items[i], i) };\n\n      } catch (error) {\n\n        results[i] = { ok: false, error };\n\n      }\n\n    }\n\n  }\n\n  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, lane));\n\n  return results;\n\n}\n\n```\nWhy it works:\n\n* **`next++` is safe without a lock.** JavaScript runs one piece of code at a time, and `const i = next++` runs to completion before any other lane resumes. Two lanes can never take the same index.\n* **Each lane waits for its own call**, then takes the next item, so there are never more than `limit` calls in flight, and a slow item doesn't hold up the other lanes.\n* **`results[i]`** writes by index, so output order matches input order even though calls finish in any order.\n* **Errors become values** (`{ ok: false, error }`). With `Promise.all`, the first rejection discards every other result, which is rarely what you want for a batch of independent jobs.\n\nUsing it:\n\n``` js title=\"Using mapLimit\"\nconst results = await mapLimit(pages, 8, (page) => ocr(page));\nconst failed = results.flatMap((r, i) => (r.ok ? [] : [{ page: pages[i], error: r.error.message }]));\njs title=\"pool.test.mjs\"\n\nimport assert from \"node:assert/strict\";\n\nconst sleep = (ms) => new Promise((r) => setTimeout(r, ms));\n\nlet active = 0;\n\nlet peak = 0;\n\nconst items = Array.from({ length: 20 }, (_, i) => i);\n\nconst results = await mapLimit(items, 4, async (n) => {\n\n  active++;\n\n  peak = Math.max(peak, active);\n\n  await sleep(5 + ((n * 7) % 11)); // uneven durations, so finishing order differs from input order\n\n  active--;\n\n  if (n === 13) throw new Error(\"boom\");\n\n  return n * n;\n\n});\n\nassert.equal(peak, 4);\n\nassert.deepEqual(\n\n  results.filter((r) => r.ok).map((r) => r.value),\n\n  items.filter((n) => n !== 13).map((n) => n * n)\n\n);\n\nassert.equal(results[13].ok, false);\n\n```\nWith `mapLimit(items, 4, …)` the peak is exactly 4; the same 20 items through `Promise.all` peak at 20. Results stay in order despite uneven durations, and item 13's failure doesn't affect the other 19. Tests for an empty list and a limit larger than the list pass too.\n\n## How do you choose the limit?\n\n* **For LLM APIs**, concurrency isn't the real limit; requests and tokens per minute are. A limit of 4–16 keeps bursts reasonable, and a [token-bucket rate limiter](https://aliahmed.si/blog/llm-rate-limiter-token-bucket-nodejs/) inside the worker enforces the provider's actual limits. Use both: the pool bounds memory and connections, the bucket bounds the rate.\n* **For your own services**, start near the number of instances behind them, measure latency as you raise it, and stop when the [p95 latency](https://aliahmed.si/blog/latency-percentiles-p95-averaging/) starts climbing: past that point you're queueing, not speeding up.\n* **For CPU-bound work** such as image processing in the same process, a pool doesn't help; JavaScript runs one thing at a time. Use worker threads.\n\n## What about retries and cancellation?\n\nKeep retries **inside the worker**, so a retried item keeps its lane and the limit still holds; [backoff with jitter](https://aliahmed.si/blog/llm-api-retries-backoff-fallbacks/) is the usual choice. To stop early, pass an `AbortSignal` to the worker and check it in `lane()` before taking the next item. If you already depend on a library such as `p-limit`, it does the same job; the point is that you don't need one for this.\n\n## Summary\n\n* `Promise.all` starts everything at once; a pool caps how many run at a time.\n* N lanes pulling from a shared index is all a pool is; `next++` is safe in single-threaded JavaScript.\n* Write results by index to keep order, and turn errors into values so one failure doesn't sink the batch.\n* Pick the limit by measuring, and pair it with a rate limiter for LLM APIs.\n\n---\n\nWritten by Ali Ahmed · [Original article](https://aliahmed.si/blog/limit-concurrent-promises-nodejs/)\n```\n\n", "url": "https://wpnews.pro/news/limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise", "canonical_source": "https://dev.to/quantumbreakz/limiting-concurrent-async-calls-in-nodejs-without-a-library-a-15-line-promise-pool-4cil", "published_at": "2026-11-19 17:36:47+00:00", "updated_at": "2026-11-19 17:43:41.416373+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools"], "entities": ["Node.js"], "also_reported_by": [], "takeaway": "A 15-line mapLimit pool using N lanes over a shared index caps in-flight calls at the limit, keeps results in input order, and records per-item errors as values instead of letting Promise.all's first rejection discard the batch.", "recommendation": {"advice": "Cap LLM batch calls with a small promise pool (limit 4–16) and add a token-bucket limiter inside the worker, since concurrency bounds connections but requests and tokens per minute bound the provider.", "playbook": "llm-cost", "playbook_url": "https://wpnews.pro/playbooks/llm-cost"}, "alternates": {"html": "https://wpnews.pro/news/limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise", "markdown": "https://wpnews.pro/news/limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise.md", "text": "https://wpnews.pro/news/limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise.txt", "jsonld": "https://wpnews.pro/news/limiting-concurrent-async-calls-in-node-js-without-a-library-a-15-line-promise.jsonld"}}