{"slug": "build-a-secure-node-js-ai-api-gateway", "title": "Build a Secure Node.js AI API Gateway", "summary": "A developer at Gate of AI published a tutorial on building a secure Node.js AI API gateway. The gateway validates JSON requests, assigns request IDs, applies rate limits, and forwards approved work to a server-side model service without exposing provider credentials to browser code. The implementation focuses on separation of concerns and application-defined controls rather than provider-specific SDKs.", "body_md": "🚀 Technical Briefing:This tutorial is part of our deep-dive series on Agentic Workflows at[Gate of AI]. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the[original article here].\n\n```\n

Build a small Node.js AI API gateway that accepts validated JSON, assigns request IDs, applies a local per-client limit, and forwards approved work to a server-side model service without exposing provider credentials to browser code.

\n\n

Prerequisites

\n\n\n

What You Are Building

\n

This tutorial builds a deliberately small Node.js AI API gateway. A client sends a JSON request to POST /api/chat. The gateway checks that the request has the expected shape, rejects oversized payloads, assigns a request ID, applies a local rate limit, and forwards the validated request to a separate model service over HTTP. The gateway then returns the model service response to the caller.

\n

The architectural idea is grounded in a commonly used separation of concerns: a web backend serves HTTP APIs while a model service performs the core predictive task. The verified context describes an example stack with a web frontend, load balancer, Node.js REST API backend, distributed task queue, and a model service. This article implements only the Node.js gateway boundary. It does not claim to prescribe a complete production architecture, a particular model vendor, or a particular model-serving framework.

\n

Keeping the gateway separate from the model service gives an application one place to apply input rules and operational controls before work reaches an AI system. It also means browser applications call your service rather than receiving a sensitive upstream credential. The exact authentication, authorization, data-retention, safety review, and deployment requirements depend on your organisation and use case, so they are intentionally not represented as universal defaults in this example.

\n

The implementation avoids unverified provider-specific SDK calls, model names, token limits, and model-response schemas. Instead, it defines a small JSON contract owned by this application. That is useful when the model service is an internal service, a separately operated inference API, or an adapter that you maintain elsewhere.

\n\n

Gateway Responsibilities and Boundaries

\n

A gateway should have a narrow job. In this tutorial it does five things: it accepts HTTP requests, parses limited-size JSON, validates required fields, records an opaque request ID, and forwards safe input to an upstream model service. It does not attempt to decide whether a generated answer is correct. It does not train a model. It does not embed keys in a frontend bundle. It also does not assume that every model service uses the same request or response structure.

\n

Our public request contract is intentionally simple:

\n
{\n```\n\n\"messages\": [\n\n{ \"role\": \"user\", \"content\": \"Explain an API gateway.\" }\n\n]\n\n}\n\nThe gateway accepts up to 20 messages, permits only `system`\n\n, `user`\n\n, and `assistant`\n\nroles, and limits each message to 12,000 characters by default. These are application controls rather than token measurements. Characters and model tokens are not the same unit, so a character limit should not be presented as a precise cost or usage limit.\n\nThe upstream service in this tutorial receives a wrapper containing a request ID and the validated messages. Its expected successful response is JSON. The gateway does not transform that JSON into a vendor-neutral completion format because no provider response format is verified in the available context. Owning a small, documented internal contract is safer than guessing at a provider API.\n\n```\n
Step 1: Create the Project and Environment File
\n
Create a new project directory. This version uses no third-party dependency, which makes the example easy to inspect and keeps its runtime surface small. The built-in Node.js HTTP server is sufficient for a focused gateway demonstration.
\n
mkdir nodejs-ai-api-gateway\n```\n\ncd nodejs-ai-api-gateway\n\nnpm init -y\n\nnpm pkg set type=module\n\nnpm pkg set scripts.start=\"node src/server.js\"\n\nnpm pkg set scripts.dev=\"node --watch src/server.js\"\n\nnpm pkg set engines.node=\">=18.0.0\"\n\nmkdir -p src\n\nCreate a `.gitignore`\n\nfile before creating local configuration. Never commit credentials or deployment-specific environment files to source control.\n\n```\nnode_modules\n.env\n.env.local\n.env.production\nlogs\ncoverage\nnpm-debug.log*\n.DS_Store\n```\n\nNext, create `.env`\n\n. `MODEL_SERVICE_URL`\n\nis the only required upstream setting. The URL must point to a service that your deployment can reach and is authorised to use. The example uses a loopback URL only as a local-development value.\n\n```\nPORT=3001\nMODEL_SERVICE_URL=http://127.0.0.1:8080/generate\nALLOWED_ORIGIN=http://localhost:3000\nMAX_BODY_BYTES=262144\nMAX_MESSAGE_CHARS=12000\nMAX_CONVERSATION_MESSAGES=20\nREQUESTS_PER_MINUTE=30\nUPSTREAM_TIMEOUT_MS=30000\n```\n\nDo not put upstream credentials in browser-exposed environment variables. If the model service requires credentials, keep them in the gateway’s server-side deployment environment and attach them only on the server. This tutorial does not include an authorization header because its name, format, and credential lifecycle are not established by the verified context.\n\n```\n
Step 2: Validate Configuration at Startup
\n
Create src/config.js. Environment variables arrive as strings, so the module explicitly parses numerical values and fails early when required configuration is invalid. Startup validation is preferable to discovering a malformed port or URL only after traffic arrives.
\n
import process from \"node:process\";\n```\n\nfunction readPositiveInteger(name, fallback, minimum, maximum) {\n\nconst raw = process.env[name] ?? String(fallback);\n\nconst value = Number.parseInt(raw, 10);\n\nif (!Number.isInteger(value) || value < minimum || value > maximum) {\n\nthrow new Error(`${name} must be an integer from ${minimum} to ${maximum}.`\n\n);\n\n}\n\nreturn value;\n\n}\n\nfunction readUrl(name) {\n\nconst raw = process.env[name];\n\nif (!raw) {\n\nthrow new Error(`${name} is required.`\n\n);\n\n}\n\ntry {\n\nreturn new URL(raw).toString();\n\n} catch {\n\nthrow new Error(`${name} must be a valid URL.`\n\n);\n\n}\n\n}\n\nexport const config = Object.freeze({\n\nport: readPositiveInteger(\"PORT\", 3001, 1, 65535),\n\nmodelServiceUrl: readUrl(\"MODEL_SERVICE_URL\"),\n\nallowedOrigin: process.env.ALLOWED_ORIGIN ?? \"[http://localhost:3000](http://localhost:3000)\",\n\nmaxBodyBytes: readPositiveInteger(\"MAX_BODY_BYTES\", 262144, 1024, 1048576),\n\nmaxMessageChars: readPositiveInteger(\"MAX_MESSAGE_CHARS\", 12000, 1, 100000),\n\nmaxConversationMessages: readPositiveInteger(\n\n\"MAX_CONVERSATION_MESSAGES\",\n\n20,\n\n1,\n\n100\n\n),\n\nrequestsPerMinute: readPositiveInteger(\n\n\"REQUESTS_PER_MINUTE\",\n\n30,\n\n1,\n\n10000\n\n),\n\nupstreamTimeoutMs: readPositiveInteger(\n\n\"UPSTREAM_TIMEOUT_MS\",\n\n30000,\n\n1000,\n\n120000\n\n)\n\n});\n\nFor local development, load environment values before starting Node. One straightforward option is to export the values in your shell. Deployment platforms should inject server-side environment variables through their own protected configuration mechanism. The code above intentionally does not rely on an unverified configuration library.\n\n```\n
Step 3: Build the Node.js Gateway
\n
Create src/server.js. The server is complete and uses only Node.js modules. It accepts POST /api/chat and GET /health. It handles preflight requests for one configured browser origin, but CORS is not authentication. A public gateway still needs an authentication and authorization design appropriate to its users.
\n
import crypto from \"node:crypto\";\n```\n\nimport http from \"node:http\";\n\nimport { config } from \"./config.js\";\n\nconst rateWindows = new Map();\n\nfunction sendJson(response, statusCode, body, requestId) {\n\nconst payload = JSON.stringify(body);\n\nresponse.writeHead(statusCode, {\n\n\"Content-Type\": \"application/json; charset=utf-8\",\n\n\"Content-Length\": Buffer.byteLength(payload),\n\n\"X-Request-Id\": requestId,\n\n\"Cache-Control\": \"no-store\"\n\n});\n\nresponse.end(payload);\n\n}\n\nfunction getRequestId(request) {\n\nconst supplied = request.headers[\"x-request-id\"];\n\nif (typeof supplied === \"string\" && supplied.length > 0 && supplied.length <= 128) {\n\nreturn supplied;\n\n}\n\nreturn crypto.randomUUID();\n\n}\n\nfunction applyCors(request, response) {\n\nconst origin = request.headers.origin;\n\nif (origin === config.allowedOrigin) {\n\nresponse.setHeader(\"Access-Control-Allow-Origin\", origin);\n\nresponse.setHeader(\"Vary\", \"Origin\");\n\nresponse.setHeader(\"Access-Control-Allow-Methods\", \"POST, GET, OPTIONS\");\n\nresponse.setHeader(\"Access-Control-Allow-Headers\", \"Content-Type, X-Request-Id\");\n\nresponse.setHeader(\"Access-Control-Max-Age\", \"86400\");\n\n}\n\n}\n\nfunction clientKey(request) {\n\nreturn request.socket.remoteAddress ?? \"unknown\";\n\n}\n\nfunction isRateLimited(request) {\n\nconst key = clientKey(request);\n\nconst now = Date.now();\n\nconst windowStart = now - 60_000;\n\nconst recent = (rateWindows.get(key) ?? []).filter((timestamp) => timestamp > windowStart);\n\nif (recent.length >= config.requestsPerMinute) {\n\nrateWindows.set(key, recent);\n\nreturn true;\n\n}\n\nrecent.push(now);\n\nrateWindows.set(key, recent);\n\nreturn false;\n\n}\n\nfunction readJsonBody(request) {\n\nreturn new Promise((resolve, reject) => {\n\nlet totalBytes = 0;\n\nconst chunks = [];\n\n```\nrequest.on(\"data\", (chunk) => {\n  totalBytes += chunk.length;\n  if (totalBytes > config.maxBodyBytes) {\n    reject(new Error(\"BODY_TOO_LARGE\"));\n    request.destroy();\n    return;\n  }\n  chunks.push(chunk);\n});\n\nrequest.on(\"end\", () => {\n  try {\n    const text = Buffer.concat(chunks).toString(\"utf8\");\n    resolve(JSON.parse(text));\n  } catch {\n    reject(new Error(\"INVALID_JSON\"));\n  }\n});\n\nrequest.on(\"error\", reject);\n```\n\n});\n\n}\n\nfunction validateChatRequest(value) {\n\nif (!value || typeof value !== \"object\" || Array.isArray(value)) {\n\nreturn \"Request body must be a JSON object.\";\n\n}\n\nif (!Array.isArray(value.messages) || value.messages.length === 0) {\n\nreturn \"messages must be a non-empty array.\";\n\n}\n\nif (value.messages.length > config.maxConversationMessages) {\n\nreturn `messages must contain at most ${config.maxConversationMessages} items.`\n\n;\n\n}\n\nconst roles = new Set([\"system\", \"user\", \"assistant\"]);\n\nlet systemCount = 0;\n\nfor (const message of value.messages) {\n\nif (!message || typeof message !== \"object\" || Array.isArray(message)) {\n\nreturn \"Each message must be an object.\";\n\n}\n\nif (!roles.has(message.role)) {\n\nreturn \"Each message role must be system, user, or assistant.\";\n\n}\n\nif (typeof message.content !== \"string\" || message.content.trim().length === 0) {\n\nreturn \"Each message content value must be a non-empty string.\";\n\n}\n\nif (message.content.length > config.maxMessageChars) {\n\nreturn `Each message content value must be at most ${config.maxMessageChars} characters.`\n\n;\n\n}\n\nif (message.role === \"system\") systemCount += 1;\n\n}\n\nif (systemCount > 1) return \"Only one system message is allowed.\";\n\nif (value.messages[0].role === \"assistant\") {\n\nreturn \"The first message cannot use the assistant role.\";\n\n}\n\nreturn null;\n\n}\n\nasync function forwardToModelService(input, requestId) {\n\nconst timeout = AbortSignal.timeout(config.upstreamTimeoutMs);\n\nconst upstreamResponse = await fetch(config.modelServiceUrl, {\n\nmethod: \"POST\",\n\nheaders: {\n\n\"Content-Type\": \"application/json\",\n\n\"X-Request-Id\": requestId\n\n},\n\nbody: JSON.stringify({ requestId, messages: input.messages }),\n\nsignal: timeout\n\n});\n\nconst contentType = upstreamResponse.headers.get(\"content-type\") ?? \"\";\n\nconst responseBody = contentType.includes(\"application/json\")\n\n? await upstreamResponse.json()\n\n: { message: await upstreamResponse.text() };\n\nreturn { status: upstreamResponse.status, body: responseBody };\n\n}\n\nconst server = http.createServer(async (request, response) => {\n\nconst requestId = getRequestId(request);\n\napplyCors(request, response);\n\nif (request.method === \"OPTIONS\") {\n\nresponse.writeHead(204, { \"X-Request-Id\": requestId });\n\nresponse.end();\n\nreturn;\n\n}\n\nif (request.method === \"GET\" && request.url === \"/health\") {\n\nsendJson(response, 200, { status: \"ok\", service: \"nodejs-ai-api-gateway\" }, requestId);\n\nreturn;\n\n}\n\nif (request.method !== \"POST\" || request.url !== \"/api/chat\") {\n\nsendJson(response, 404, { error: { code: \"NOT_FOUND\", message: \"Route not found.\" } }, requestId);\n\nreturn;\n\n}\n\nif (isRateLimited(request)) {\n\nsendJson(response, 429, { error: { code: \"RATE_LIMITED\", message: \"Try again shortly.\" } }, requestId);\n\nreturn;\n\n}\n\nif (!request.headers[\"content-type\"]?.includes(\"application/json\")) {\n\nsendJson(response, 415, { error: { code: \"UNSUPPORTED_MEDIA_TYPE\", message: \"Use application/json.\" } }, requestId);\n\nreturn;\n\n}\n\ntry {\n\nconst input = await readJsonBody(request);\n\nconst validationError = validateChatRequest(input);\n\n```\nif (validationError) {\n  sendJson(response, 400, { error: { code: \"INVALID_REQUEST\", message: validationError } }, requestId);\n  return;\n}\n\nconst upstream = await forwardToModelService(input, requestId);\nsendJson(response, upstream.status, { requestId, data: upstream.body }, requestId);\n```\n\n} catch (error) {\n\nif (error.message === \"BODY_TOO_LARGE\") {\n\nsendJson(response, 413, { error: { code: \"BODY_TOO_LARGE\", message: \"Request body exceeds the configured limit.\" } }, requestId);\n\nreturn;\n\n}\n\n```\nif (error.message === \"INVALID_JSON\") {\n  sendJson(response, 400, { error: { code: \"INVALID_JSON\", message: \"Request body must contain valid JSON.\" } }, requestId);\n  return;\n}\n\nconsole.error(JSON.stringify({ requestId, error: String(error) }));\nsendJson(response, 502, { error: { code: \"MODEL_SERVICE_ERROR\", message: \"The model service could not be reached or did not complete the request.\" } }, requestId);\n```\n\n}\n\n});\n\nserver.listen(config.port, () => {\n\nconsole.log(JSON.stringify({ event: \"listening\", port: config.port }));\n\n});\n\nfunction shutdown(signal) {\n\nconsole.log(JSON.stringify({ event: \"shutdown_started\", signal }));\n\nserver.close(() => process.exit(0));\n\nsetTimeout(() => process.exit(1), 10_000).unref();\n\n}\n\nprocess.on(\"SIGINT\", () => shutdown(\"SIGINT\"));\n\nprocess.on(\"SIGTERM\", () => shutdown(\"SIGTERM\"));\n\nThe local rate limiter uses process memory and the socket address as its key. This is suitable only for a single-process demonstration. When a service runs behind a load balancer or on multiple instances, each process has its own memory and may observe a different client address. Use a shared, deliberately designed rate-limit mechanism and authenticated identity signals before treating limits as an enforceable organisation-wide control.\n\nThe server logs only an opaque request ID and an error string in the catch path. It does not intentionally log message content. This is a practical default because prompts may contain sensitive business information, source code, or personal information. If you need content logging for a defined evaluation process, decide separately what may be retained, who may access it, and how deletion and incident response work.\n\n```\n
Step 4: Run and Test the Gateway
\n
Set the environment variables in your shell, then start the server. The exact shell syntax differs by operating system. On a Unix-like shell, the following starts the application with the example local URL.
\n
export PORT=3001\n```\n\nexport MODEL_SERVICE_URL=[http://127.0.0.1:8080/generate](http://127.0.0.1:8080/generate)\n\nexport ALLOWED_ORIGIN=[http://localhost:3000](http://localhost:3000)\n\nnpm run dev\n\nFirst test the health endpoint. It does not call the model service.\n\n```\ncurl --include http://localhost:3001/health\n```\n\nThen send a valid chat request. This request will reach the configured model service, so it can succeed only when that service is available and implements the contract expected by your application.\n\n```\ncurl --include --request POST http://localhost:3001/api/chat \\\n  --header \"Content-Type: application/json\" \\\n  --data '{\n    \"messages\": [\n      { \"role\": \"system\", \"content\": \"Answer concisely.\" },\n      { \"role\": \"user\", \"content\": \"What is an API gateway?\" }\n    ]\n  }'\n```\n\nAlso test negative paths. Send malformed JSON to confirm the gateway returns `400`\n\n. Send a request with an invalid role to verify the application-level schema. Finally, temporarily stop the model service to check the gateway’s `502`\n\nresponse. Failure testing is essential because network availability and upstream behaviour are not guaranteed.\n\n```\ncurl --include --request POST http://localhost:3001/api/chat \\\n  --header \"Content-Type: application/json\" \\\n  --data '{\"messages\":[{\"role\":\"unknown\",\"content\":\"Hello\"}]}'\n
What to Build Next
\n
Start with authentication. CORS controls which compliant browsers may read a response; it does not establish who is allowed to use an endpoint. Add an authentication mechanism appropriate to your application, then associate requests with a user, service, organisation, or tenant before applying meaningful quotas.
\n
Next, define the model-service contract explicitly. Document the accepted request body, expected response body, timeout behaviour, error statuses, and versioning strategy. If you introduce a queue for long-running work, make the request lifecycle explicit rather than pretending every task will complete during one synchronous HTTP call. The verified context’s example architecture includes a distributed task queue alongside a Node.js backend and model service, which is a useful direction to evaluate for workloads that do not fit a simple request-response interaction.
\n
For scale, put the gateway behind infrastructure that your team operates and monitors. The verified context describes a load balancer as part of an example production stack. If you deploy multiple gateway instances, replace the in-memory limiter with a shared approach, ensure request IDs flow through every service, and test how client address information is handled by your proxy configuration.
\n
For GCC and Middle East deployments, avoid unsupported assumptions about local hosting, legal requirements, or named government programmes. Instead, have legal, security, and data-governance stakeholders assess where data is processed, what information is sent to the model service, how long records are retained, and which regional contractual or regulatory obligations apply to the specific organisation. A gateway boundary makes those controls easier to place consistently, but it does not itself establish compliance.
\n
Finally, keep dependencies and runtime components patched. The verified context includes a National Vulnerability Database entry, reinforcing the practical need to monitor security advisories relevant to the software you operate. Review advisories, test upgrades, and maintain a repeatable deployment process. A compact gateway with a clear contract is easier to test, audit, and evolve than provider-specific calls scattered across browser and backend applications.
\n```\n\n", "url": "https://wpnews.pro/news/build-a-secure-node-js-ai-api-gateway", "canonical_source": "https://dev.to/gateofai/build-a-secure-nodejs-ai-api-gateway-5503", "published_at": "2026-08-13 18:13:39+00:00", "updated_at": "2026-08-13 18:48:10.943075+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure", "ai-products"], "entities": ["Gate of AI", "Node.js"], "alternates": {"html": "https://wpnews.pro/news/build-a-secure-node-js-ai-api-gateway", "markdown": "https://wpnews.pro/news/build-a-secure-node-js-ai-api-gateway.md", "text": "https://wpnews.pro/news/build-a-secure-node-js-ai-api-gateway.txt", "jsonld": "https://wpnews.pro/news/build-a-secure-node-js-ai-api-gateway.jsonld"}}