Build a Secure Node.js AI API Gateway

A developer at Gate of AI published a tutorial on building a secure Node.js AI API gateway. The gateway validates JSON requests, assigns request IDs, applies rate limits, and forwards approved work to a server-side model service without exposing provider credentials to browser code. The implementation focuses on separation of concerns and application-defined controls rather than provider-specific SDKs.

🚀 Technical Briefing:This tutorial is part of our deep-dive series on Agentic Workflows at Gate of AI . For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the original article here . <p Build a small Node.js AI API gateway that accepts validated JSON, assigns request IDs, applies a local per-client limit, and forwards approved work to a server-side model service without exposing provider credentials to browser code.</p <h2 Prerequisites</h2 <ul <li Node.js 18 or later. This tutorial uses the runtime’s built-in <code http</code , <code crypto</code , and <code fetch</code capabilities.</li <li Basic familiarity with JavaScript, JSON, HTTP requests, environment variables, and a terminal.</li <li A model service that your gateway is permitted to call over HTTP. The gateway does not contain a model or a model-provider SDK.</li <li A REST client such as <code curl</code , Postman, Bruno, Insomnia, or the VS Code REST Client extension.</li </ul <h2 What You Are Building</h2 <p This tutorial builds a deliberately small Node.js AI API gateway. A client sends a JSON request to <code POST /api/chat</code . The gateway checks that the request has the expected shape, rejects oversized payloads, assigns a request ID, applies a local rate limit, and forwards the validated request to a separate model service over HTTP. The gateway then returns the model service response to the caller.</p <p The architectural idea is grounded in a commonly used separation of concerns: a web backend serves HTTP APIs while a model service performs the core predictive task. The verified context describes an example stack with a web frontend, load balancer, Node.js REST API backend, distributed task queue, and a model service. This article implements only the Node.js gateway boundary. It does not claim to prescribe a complete production architecture, a particular model vendor, or a particular model-serving framework.</p <p Keeping the gateway separate from the model service gives an application one place to apply input rules and operational controls before work reaches an AI system. It also means browser applications call your service rather than receiving a sensitive upstream credential. The exact authentication, authorization, data-retention, safety review, and deployment requirements depend on your organisation and use case, so they are intentionally not represented as universal defaults in this example.</p <p The implementation avoids unverified provider-specific SDK calls, model names, token limits, and model-response schemas. Instead, it defines a small JSON contract owned by this application. That is useful when the model service is an internal service, a separately operated inference API, or an adapter that you maintain elsewhere.</p <h2 Gateway Responsibilities and Boundaries</h2 <p A gateway should have a narrow job. In this tutorial it does five things: it accepts HTTP requests, parses limited-size JSON, validates required fields, records an opaque request ID, and forwards safe input to an upstream model service. It does not attempt to decide whether a generated answer is correct. It does not train a model. It does not embed keys in a frontend bundle. It also does not assume that every model service uses the same request or response structure.</p <p Our public request contract is intentionally simple:</p <pre <code { "messages": { "role": "user", "content": "Explain an API gateway." } } The gateway accepts up to 20 messages, permits only system , user , and assistant roles, and limits each message to 12,000 characters by default. These are application controls rather than token measurements. Characters and model tokens are not the same unit, so a character limit should not be presented as a precise cost or usage limit. The upstream service in this tutorial receives a wrapper containing a request ID and the validated messages. Its expected successful response is JSON. The gateway does not transform that JSON into a vendor-neutral completion format because no provider response format is verified in the available context. Owning a small, documented internal contract is safer than guessing at a provider API. <h2 Step 1: Create the Project and Environment File</h2 <p Create a new project directory. This version uses no third-party dependency, which makes the example easy to inspect and keeps its runtime surface small. The built-in Node.js HTTP server is sufficient for a focused gateway demonstration.</p <pre <code mkdir nodejs-ai-api-gateway cd nodejs-ai-api-gateway npm init -y npm pkg set type=module npm pkg set scripts.start="node src/server.js" npm pkg set scripts.dev="node --watch src/server.js" npm pkg set engines.node=" =18.0.0" mkdir -p src Create a .gitignore file before creating local configuration. Never commit credentials or deployment-specific environment files to source control. node modules .env .env.local .env.production logs coverage npm-debug.log .DS Store Next, create .env . MODEL SERVICE URL is the only required upstream setting. The URL must point to a service that your deployment can reach and is authorised to use. The example uses a loopback URL only as a local-development value. PORT=3001 MODEL SERVICE URL=http://127.0.0.1:8080/generate ALLOWED ORIGIN=http://localhost:3000 MAX BODY BYTES=262144 MAX MESSAGE CHARS=12000 MAX CONVERSATION MESSAGES=20 REQUESTS PER MINUTE=30 UPSTREAM TIMEOUT MS=30000 Do not put upstream credentials in browser-exposed environment variables. If the model service requires credentials, keep them in the gateway’s server-side deployment environment and attach them only on the server. This tutorial does not include an authorization header because its name, format, and credential lifecycle are not established by the verified context. <h2 Step 2: Validate Configuration at Startup</h2 <p Create <code src/config.js</code . Environment variables arrive as strings, so the module explicitly parses numerical values and fails early when required configuration is invalid. Startup validation is preferable to discovering a malformed port or URL only after traffic arrives.</p <pre <code import process from "node:process"; function readPositiveInteger name, fallback, minimum, maximum { const raw = process.env name ?? String fallback ; const value = Number.parseInt raw, 10 ; if Number.isInteger value || value < minimum || value maximum { throw new Error ${name} must be an integer from ${minimum} to ${maximum}. ; } return value; } function readUrl name { const raw = process.env name ; if raw { throw new Error ${name} is required. ; } try { return new URL raw .toString ; } catch { throw new Error ${name} must be a valid URL. ; } } export const config = Object.freeze { port: readPositiveInteger "PORT", 3001, 1, 65535 , modelServiceUrl: readUrl "MODEL SERVICE URL" , allowedOrigin: process.env.ALLOWED ORIGIN ?? " http://localhost:3000 http://localhost:3000 ", maxBodyBytes: readPositiveInteger "MAX BODY BYTES", 262144, 1024, 1048576 , maxMessageChars: readPositiveInteger "MAX MESSAGE CHARS", 12000, 1, 100000 , maxConversationMessages: readPositiveInteger "MAX CONVERSATION MESSAGES", 20, 1, 100 , requestsPerMinute: readPositiveInteger "REQUESTS PER MINUTE", 30, 1, 10000 , upstreamTimeoutMs: readPositiveInteger "UPSTREAM TIMEOUT MS", 30000, 1000, 120000 } ; For local development, load environment values before starting Node. One straightforward option is to export the values in your shell. Deployment platforms should inject server-side environment variables through their own protected configuration mechanism. The code above intentionally does not rely on an unverified configuration library. <h2 Step 3: Build the Node.js Gateway</h2 <p Create <code src/server.js</code . The server is complete and uses only Node.js modules. It accepts <code POST /api/chat</code and <code GET /health</code . It handles preflight requests for one configured browser origin, but CORS is not authentication. A public gateway still needs an authentication and authorization design appropriate to its users.</p <pre <code import crypto from "node:crypto"; import http from "node:http"; import { config } from "./config.js"; const rateWindows = new Map ; function sendJson response, statusCode, body, requestId { const payload = JSON.stringify body ; response.writeHead statusCode, { "Content-Type": "application/json; charset=utf-8", "Content-Length": Buffer.byteLength payload , "X-Request-Id": requestId, "Cache-Control": "no-store" } ; response.end payload ; } function getRequestId request { const supplied = request.headers "x-request-id" ; if typeof supplied === "string" && supplied.length 0 && supplied.length <= 128 { return supplied; } return crypto.randomUUID ; } function applyCors request, response { const origin = request.headers.origin; if origin === config.allowedOrigin { response.setHeader "Access-Control-Allow-Origin", origin ; response.setHeader "Vary", "Origin" ; response.setHeader "Access-Control-Allow-Methods", "POST, GET, OPTIONS" ; response.setHeader "Access-Control-Allow-Headers", "Content-Type, X-Request-Id" ; response.setHeader "Access-Control-Max-Age", "86400" ; } } function clientKey request { return request.socket.remoteAddress ?? "unknown"; } function isRateLimited request { const key = clientKey request ; const now = Date.now ; const windowStart = now - 60 000; const recent = rateWindows.get key ?? .filter timestamp = timestamp windowStart ; if recent.length = config.requestsPerMinute { rateWindows.set key, recent ; return true; } recent.push now ; rateWindows.set key, recent ; return false; } function readJsonBody request { return new Promise resolve, reject = { let totalBytes = 0; const chunks = ; request.on "data", chunk =&gt; { totalBytes += chunk.length; if totalBytes &gt; config.maxBodyBytes { reject new Error "BODY TOO LARGE" ; request.destroy ; return; } chunks.push chunk ; } ; request.on "end", =&gt; { try { const text = Buffer.concat chunks .toString "utf8" ; resolve JSON.parse text ; } catch { reject new Error "INVALID JSON" ; } } ; request.on "error", reject ; } ; } function validateChatRequest value { if value || typeof value == "object" || Array.isArray value { return "Request body must be a JSON object."; } if Array.isArray value.messages || value.messages.length === 0 { return "messages must be a non-empty array."; } if value.messages.length config.maxConversationMessages { return messages must contain at most ${config.maxConversationMessages} items. ; } const roles = new Set "system", "user", "assistant" ; let systemCount = 0; for const message of value.messages { if message || typeof message == "object" || Array.isArray message { return "Each message must be an object."; } if roles.has message.role { return "Each message role must be system, user, or assistant."; } if typeof message.content == "string" || message.content.trim .length === 0 { return "Each message content value must be a non-empty string."; } if message.content.length config.maxMessageChars { return Each message content value must be at most ${config.maxMessageChars} characters. ; } if message.role === "system" systemCount += 1; } if systemCount 1 return "Only one system message is allowed."; if value.messages 0 .role === "assistant" { return "The first message cannot use the assistant role."; } return null; } async function forwardToModelService input, requestId { const timeout = AbortSignal.timeout config.upstreamTimeoutMs ; const upstreamResponse = await fetch config.modelServiceUrl, { method: "POST", headers: { "Content-Type": "application/json", "X-Request-Id": requestId }, body: JSON.stringify { requestId, messages: input.messages } , signal: timeout } ; const contentType = upstreamResponse.headers.get "content-type" ?? ""; const responseBody = contentType.includes "application/json" ? await upstreamResponse.json : { message: await upstreamResponse.text }; return { status: upstreamResponse.status, body: responseBody }; } const server = http.createServer async request, response = { const requestId = getRequestId request ; applyCors request, response ; if request.method === "OPTIONS" { response.writeHead 204, { "X-Request-Id": requestId } ; response.end ; return; } if request.method === "GET" && request.url === "/health" { sendJson response, 200, { status: "ok", service: "nodejs-ai-api-gateway" }, requestId ; return; } if request.method == "POST" || request.url == "/api/chat" { sendJson response, 404, { error: { code: "NOT FOUND", message: "Route not found." } }, requestId ; return; } if isRateLimited request { sendJson response, 429, { error: { code: "RATE LIMITED", message: "Try again shortly." } }, requestId ; return; } if request.headers "content-type" ?.includes "application/json" { sendJson response, 415, { error: { code: "UNSUPPORTED MEDIA TYPE", message: "Use application/json." } }, requestId ; return; } try { const input = await readJsonBody request ; const validationError = validateChatRequest input ; if validationError { sendJson response, 400, { error: { code: "INVALID REQUEST", message: validationError } }, requestId ; return; } const upstream = await forwardToModelService input, requestId ; sendJson response, upstream.status, { requestId, data: upstream.body }, requestId ; } catch error { if error.message === "BODY TOO LARGE" { sendJson response, 413, { error: { code: "BODY TOO LARGE", message: "Request body exceeds the configured limit." } }, requestId ; return; } if error.message === "INVALID JSON" { sendJson response, 400, { error: { code: "INVALID JSON", message: "Request body must contain valid JSON." } }, requestId ; return; } console.error JSON.stringify { requestId, error: String error } ; sendJson response, 502, { error: { code: "MODEL SERVICE ERROR", message: "The model service could not be reached or did not complete the request." } }, requestId ; } } ; server.listen config.port, = { console.log JSON.stringify { event: "listening", port: config.port } ; } ; function shutdown signal { console.log JSON.stringify { event: "shutdown started", signal } ; server.close = process.exit 0 ; setTimeout = process.exit 1 , 10 000 .unref ; } process.on "SIGINT", = shutdown "SIGINT" ; process.on "SIGTERM", = shutdown "SIGTERM" ; The local rate limiter uses process memory and the socket address as its key. This is suitable only for a single-process demonstration. When a service runs behind a load balancer or on multiple instances, each process has its own memory and may observe a different client address. Use a shared, deliberately designed rate-limit mechanism and authenticated identity signals before treating limits as an enforceable organisation-wide control. The server logs only an opaque request ID and an error string in the catch path. It does not intentionally log message content. This is a practical default because prompts may contain sensitive business information, source code, or personal information. If you need content logging for a defined evaluation process, decide separately what may be retained, who may access it, and how deletion and incident response work. <h2 Step 4: Run and Test the Gateway</h2 <p Set the environment variables in your shell, then start the server. The exact shell syntax differs by operating system. On a Unix-like shell, the following starts the application with the example local URL.</p <pre <code export PORT=3001 export MODEL SERVICE URL= http://127.0.0.1:8080/generate http://127.0.0.1:8080/generate export ALLOWED ORIGIN= http://localhost:3000 http://localhost:3000 npm run dev First test the health endpoint. It does not call the model service. curl --include http://localhost:3001/health Then send a valid chat request. This request will reach the configured model service, so it can succeed only when that service is available and implements the contract expected by your application. curl --include --request POST http://localhost:3001/api/chat \ --header "Content-Type: application/json" \ --data '{ "messages": { "role": "system", "content": "Answer concisely." }, { "role": "user", "content": "What is an API gateway?" } }' Also test negative paths. Send malformed JSON to confirm the gateway returns 400 . Send a request with an invalid role to verify the application-level schema. Finally, temporarily stop the model service to check the gateway’s 502 response. Failure testing is essential because network availability and upstream behaviour are not guaranteed. curl --include --request POST http://localhost:3001/api/chat \ --header "Content-Type: application/json" \ --data '{"messages": {"role":"unknown","content":"Hello"} }' <h2 What to Build Next</h2 <p Start with authentication. CORS controls which compliant browsers may read a response; it does not establish who is allowed to use an endpoint. Add an authentication mechanism appropriate to your application, then associate requests with a user, service, organisation, or tenant before applying meaningful quotas.</p <p Next, define the model-service contract explicitly. Document the accepted request body, expected response body, timeout behaviour, error statuses, and versioning strategy. If you introduce a queue for long-running work, make the request lifecycle explicit rather than pretending every task will complete during one synchronous HTTP call. The verified context’s example architecture includes a distributed task queue alongside a Node.js backend and model service, which is a useful direction to evaluate for workloads that do not fit a simple request-response interaction.</p <p For scale, put the gateway behind infrastructure that your team operates and monitors. The verified context describes a load balancer as part of an example production stack. If you deploy multiple gateway instances, replace the in-memory limiter with a shared approach, ensure request IDs flow through every service, and test how client address information is handled by your proxy configuration.</p <p For GCC and Middle East deployments, avoid unsupported assumptions about local hosting, legal requirements, or named government programmes. Instead, have legal, security, and data-governance stakeholders assess where data is processed, what information is sent to the model service, how long records are retained, and which regional contractual or regulatory obligations apply to the specific organisation. A gateway boundary makes those controls easier to place consistently, but it does not itself establish compliance.</p <p Finally, keep dependencies and runtime components patched. The verified context includes a National Vulnerability Database entry, reinforcing the practical need to monitor security advisories relevant to the software you operate. Review advisories, test upgrades, and maintain a repeatable deployment process. A compact gateway with a clear contract is easier to test, audit, and evolve than provider-specific calls scattered across browser and backend applications.</p