{"slug": "api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out", "title": "API Gateway Payload Routing vs. Lambda Invocation Overhead: The Request Fan-Out Cost Nobody Budgets", "summary": "An engineer's analysis of AWS Lambda fan-out patterns shows that API Gateway's control-plane scheduling hop, typically 1–4 ms but exceeding 20 ms under burst, is invisible in CloudWatch REPORT lines and compounds across parallel downstream invocations. The writeup argues teams underprovision downstream functions, since pipeline concurrency is capped at min(classifier_concurrency, downstream_concurrency), and recommends provisioning to N × fan-out-width for peak concurrent requests.", "body_md": "Most Lambda cost models stop at GB-seconds and invocation count. That framing collapses when a single inbound request triggers multiple downstream Lambda invocations—fan-out patterns common in event enrichment pipelines, multi-tenant authorization checks, and AI tool-call dispatchers. The budget hole isn't the compute; it's the orchestration overhead layered between API Gateway and your function fleet, compounded by how Go's execution model interacts with Lambda's initialization contract.\n\nAn API Gateway HTTP API (not REST API) routes a request to a Lambda function. That round-trip includes:\n\nThe control plane hop in step 3 is the one engineers routinely ignore. Under normal load it adds 1–4 ms. Under burst—when the Lambda service is simultaneously allocating new execution environments for concurrent requests—that number climbs past 20 ms, invisibly, because it doesn't appear in your function's own duration metric. `REPORT` lines in CloudWatch show billed duration inside the environment. They do not show control plane scheduling latency.\n\nFor a single-invocation path this is tolerable. For fan-out it multiplies.\n\nConsider a request classification pipeline: an inbound HTTP POST hits API Gateway, which invokes a Go Lambda (`classifier`) that parses the payload and dispatches to three downstream Lambdas in parallel—`enricher`, `policy-checker`, and `audit-writer`. The caller waits for the classifier to aggregate responses.\n\n```\nClient → APIGW → classifier (Go Lambda)\n                    ├─ invoke → enricher\n                    ├─ invoke → policy-checker\n                    └─ invoke → audit-writer\n```\n\nThe classifier uses `sdk-go-v2` to issue parallel `InvokeWithContext` calls:\n\n```\nfunc fanOut(ctx context.Context, client *lambda.Client, payload []byte) ([][]byte, error) {\n    targets := []string{\"enricher\", \"policy-checker\", \"audit-writer\"}\n    results := make([][]byte, len(targets))\n    errs := make([]error, len(targets))\n\n    var wg sync.WaitGroup\n    for i, fn := range targets {\n        wg.Add(1)\n        go func(idx int, name string) {\n            defer wg.Done()\n            out, err := client.Invoke(ctx, &lambda.InvokeInput{\n                FunctionName:   aws.String(name),\n                InvocationType: types.InvocationTypeRequestResponse,\n                Payload:        payload,\n            })\n            if err != nil {\n                errs[idx] = err\n                return\n            }\n            results[idx] = out.Payload\n        }(i, fn)\n    }\n    wg.Wait()\n    // aggregate errors\n    for _, e := range errs {\n        if e != nil {\n            return nil, e\n        }\n    }\n    return results, nil\n}\n```\n\nThe latency of the classifier is now `max(enricher, policy-checker, audit-writer)` plus the classifier's own work. Each downstream invocation carries its own control plane scheduling cost. If any downstream function is cold—because concurrent load just exhausted provisioned concurrency—the whole request stalls on that slowest tail.\n\nCost is also additive: three downstream synchronous invocations billed at their individual durations, plus the classifier held open waiting. At 10 RPS with average 200 ms downstream functions, you're running roughly 6,000 invocations per minute across four functions. At scale, this compounds.\n\nProvisioned concurrency is budgeted per function. If you provision 10 warm environments for `policy-checker`, that covers 10 simultaneous classifier calls hitting that function. But the fan-out means each classifier invocation drives one `policy-checker` invocation. Your actual concurrency ceiling for the whole pipeline is `min(classifier_concurrency, policy-checker_concurrency)`. Teams typically provision the gateway-facing Lambda and forget the downstream fleet, then wonder why p99 blows up under burst.\n\nThe correct model: provision concurrency to `N × fan-out-width` where N is your peak expected concurrent classifier invocations. If you expect 50 simultaneous inbound requests and fan-out width is 3, each downstream function needs 50 provisioned environments, and the classifier needs 50. That's 200 provisioned environments total—a budget line that doesn't appear in naive per-function cost estimates.\n\nIf the caller doesn't need the downstream results synchronously, the architecture should change:\n\n```\nClient → APIGW → classifier (Go Lambda)\n                    └─ publish → SQS FIFO or Standard queue\n                                    ├─ enricher (Lambda, event source mapping)\n                                    ├─ policy-checker (Lambda, event source mapping)\n                                    └─ audit-writer (Lambda, event source mapping)\n```\n\nThe classifier returns immediately after publishing the SQS message. Downstream Lambdas are triggered by event source mappings. Benefits:\n\nThe tradeoff: the caller gets a correlation ID, not results. Responses must be pushed via WebSocket, polling, or a callback. For audit-writing and enrichment that feed async stores, this is almost always the correct choice. For policy-checking that gates the response, it is not.\n\nREST API Gateway adds per-request overhead for request/response mapping, usage plans, and WAF evaluation that HTTP API does not. For internal service-to-service fan-out invocations, REST API Gateway is the wrong choice—its feature set (API keys, caching, request transformation) is useful at the public edge, not between services.\n\nFor ECS-to-Lambda fan-out, a direct SDK invoke is cheaper than routing through API Gateway entirely. API Gateway makes sense at the public ingress boundary. Internal fan-out from a Lambda or ECS task should call Lambda directly via the AWS SDK or publish to SQS/SNS—not round-trip through another API Gateway stage.\n\nALB as a Lambda trigger deserves mention: ALB-to-Lambda costs $0.008 per LCU versus API Gateway HTTP API at $1.00 per million requests. For high-volume internal traffic, ALB fronting Lambda is materially cheaper, at the cost of losing API Gateway's request routing, auth integration, and usage plan machinery.\n\nGo Lambda functions use the `provided.al2023` runtime. The execution environment is a Firecracker microVM. Go binaries start fast—typically 50–150 ms cold start for a well-built binary—but that figure is invalidated by:\n\nKeep the binary lean. Initialize all SDK clients—`lambda.NewFromConfig`, `sqs.NewFromConfig`—in `main()` before `lambda.Start()`. The execution environment reuses the initialized state across warm invocations. The handler itself should be allocation-light on the hot path; escape-analysis-driven heap allocation in a high-RPS handler adds GC pressure that shows up in duration metrics.\n\nFor fan-out handlers, reuse the `*lambda.Client` across invocations. The SDK's HTTP transport maintains a connection pool to the Lambda service endpoint. Reconstructing the client per invocation destroys that pool and forces fresh TCP+TLS handshakes on every fan-out call.\n\nBefore committing to a Lambda fan-out pattern, answer these:\n\n**1. Does the caller need downstream results synchronously?**\n\nIf no: SQS-mediated async fan-out. Classifier publishes and returns. Downstream Lambdas consume from queue.\n\n**2. What is the fan-out width and expected peak concurrency?**\n\nIf `width × peak_concurrency > 200`: provisioned concurrency costs will surprise you. Model the full fleet, not the gateway function alone.\n\n**3. Is the downstream latency bounded and consistent?**\n\nIf downstream p99 >> p50: synchronous fan-out will be dominated by tail latency. Consider circuit breaking with `context.WithTimeout` per downstream call and a fallback rather than waiting for the slowest.\n\n**4. Is this internal or public traffic?**\n\nInternal: skip API Gateway, use direct SDK invoke or SQS. Public edge: API Gateway HTTP API unless you need REST API features.\n\n**5. What is the retry behavior under partial failure?**\n\nSynchronous fan-out with all-or-nothing aggregation means one downstream failure fails the whole request. If `audit-writer` is non-critical, its failure should not degrade the response. Model failure modes per downstream function and decide whether partial success is acceptable before writing the aggregation logic.\n\nThe GB-seconds on your Lambda cost dashboard are a trailing indicator. The architectural decisions—sync vs. async fan-out, API Gateway tier, provisioned concurrency fleet sizing, client initialization discipline in Go—determine whether that trailing indicator is tolerable or a monthly surprise.", "url": "https://wpnews.pro/news/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out", "canonical_source": "https://dev.to/neeraj_singhi_golang/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out-cost-nobody-budgets-5e01", "published_at": "2026-11-14 03:38:00+00:00", "updated_at": "2026-11-14 03:50:47.558055+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops", "developer-tools"], "entities": ["AWS", "AWS Lambda", "Amazon API Gateway", "CloudWatch", "Go"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out", "markdown": "https://wpnews.pro/news/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out.md", "text": "https://wpnews.pro/news/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out.txt", "jsonld": "https://wpnews.pro/news/api-gateway-payload-routing-vs-lambda-invocation-overhead-the-request-fan-out.jsonld"}}