# Build a Secure Node.js AI API Gateway > Source: > Published: 2026-08-13 18:13:39+00:00 🚀 Technical Briefing:This tutorial is part of our deep-dive series on Agentic Workflows at[Gate of AI]. For the full technical breakdown, interactive code sandbox, and the native Arabic translation, visit the[original article here]. ```

Build a small Node.js AI API gateway that accepts validated JSON, assigns request IDs, applies a local per-client limit, and forwards approved work to a server-side model service without exposing provider credentials to browser code.

Prerequisites

What You Are Building

This tutorial builds a deliberately small Node.js AI API gateway. A client sends a JSON request to POST /api/chat. The gateway checks that the request has the expected shape, rejects oversized payloads, assigns a request ID, applies a local rate limit, and forwards the validated request to a separate model service over HTTP. The gateway then returns the model service response to the caller.

The architectural idea is grounded in a commonly used separation of concerns: a web backend serves HTTP APIs while a model service performs the core predictive task. The verified context describes an example stack with a web frontend, load balancer, Node.js REST API backend, distributed task queue, and a model service. This article implements only the Node.js gateway boundary. It does not claim to prescribe a complete production architecture, a particular model vendor, or a particular model-serving framework.

Keeping the gateway separate from the model service gives an application one place to apply input rules and operational controls before work reaches an AI system. It also means browser applications call your service rather than receiving a sensitive upstream credential. The exact authentication, authorization, data-retention, safety review, and deployment requirements depend on your organisation and use case, so they are intentionally not represented as universal defaults in this example.

The implementation avoids unverified provider-specific SDK calls, model names, token limits, and model-response schemas. Instead, it defines a small JSON contract owned by this application. That is useful when the model service is an internal service, a separately operated inference API, or an adapter that you maintain elsewhere.

Gateway Responsibilities and Boundaries

A gateway should have a narrow job. In this tutorial it does five things: it accepts HTTP requests, parses limited-size JSON, validates required fields, records an opaque request ID, and forwards safe input to an upstream model service. It does not attempt to decide whether a generated answer is correct. It does not train a model. It does not embed keys in a frontend bundle. It also does not assume that every model service uses the same request or response structure.

Our public request contract is intentionally simple:

{
```

"messages": [

{ "role": "user", "content": "Explain an API gateway." }

]

}

The gateway accepts up to 20 messages, permits only `system`

, `user`

, and `assistant`

roles, and limits each message to 12,000 characters by default. These are application controls rather than token measurements. Characters and model tokens are not the same unit, so a character limit should not be presented as a precise cost or usage limit.

The upstream service in this tutorial receives a wrapper containing a request ID and the validated messages. Its expected successful response is JSON. The gateway does not transform that JSON into a vendor-neutral completion format because no provider response format is verified in the available context. Owning a small, documented internal contract is safer than guessing at a provider API.

```

Step 1: Create the Project and Environment File


Create a new project directory. This version uses no third-party dependency, which makes the example easy to inspect and keeps its runtime surface small. The built-in Node.js HTTP server is sufficient for a focused gateway demonstration.


mkdir nodejs-ai-api-gateway
```

cd nodejs-ai-api-gateway

npm init -y

npm pkg set type=module

npm pkg set scripts.start="node src/server.js"

npm pkg set scripts.dev="node --watch src/server.js"

npm pkg set engines.node=">=18.0.0"

mkdir -p src

Create a `.gitignore`

file before creating local configuration. Never commit credentials or deployment-specific environment files to source control.

```
node_modules
.env
.env.local
.env.production
logs
coverage
npm-debug.log*
.DS_Store
```

Next, create `.env`

. `MODEL_SERVICE_URL`

is the only required upstream setting. The URL must point to a service that your deployment can reach and is authorised to use. The example uses a loopback URL only as a local-development value.

```
PORT=3001
MODEL_SERVICE_URL=http://127.0.0.1:8080/generate
ALLOWED_ORIGIN=http://localhost:3000
MAX_BODY_BYTES=262144
MAX_MESSAGE_CHARS=12000
MAX_CONVERSATION_MESSAGES=20
REQUESTS_PER_MINUTE=30
UPSTREAM_TIMEOUT_MS=30000
```

Do not put upstream credentials in browser-exposed environment variables. If the model service requires credentials, keep them in the gateway’s server-side deployment environment and attach them only on the server. This tutorial does not include an authorization header because its name, format, and credential lifecycle are not established by the verified context.

```

Step 2: Validate Configuration at Startup


Create src/config.js. Environment variables arrive as strings, so the module explicitly parses numerical values and fails early when required configuration is invalid. Startup validation is preferable to discovering a malformed port or URL only after traffic arrives.


import process from "node:process";
```

function readPositiveInteger(name, fallback, minimum, maximum) {

const raw = process.env[name] ?? String(fallback);

const value = Number.parseInt(raw, 10);

if (!Number.isInteger(value) || value < minimum || value > maximum) {

throw new Error(`${name} must be an integer from ${minimum} to ${maximum}.`

);

}

return value;

}

function readUrl(name) {

const raw = process.env[name];

if (!raw) {

throw new Error(`${name} is required.`

);

}

try {

return new URL(raw).toString();

} catch {

throw new Error(`${name} must be a valid URL.`

);

}

}

export const config = Object.freeze({

port: readPositiveInteger("PORT", 3001, 1, 65535),

modelServiceUrl: readUrl("MODEL_SERVICE_URL"),

allowedOrigin: process.env.ALLOWED_ORIGIN ?? "[http://localhost:3000](http://localhost:3000)",

maxBodyBytes: readPositiveInteger("MAX_BODY_BYTES", 262144, 1024, 1048576),

maxMessageChars: readPositiveInteger("MAX_MESSAGE_CHARS", 12000, 1, 100000),

maxConversationMessages: readPositiveInteger(

"MAX_CONVERSATION_MESSAGES",

20,

1,

100

),

requestsPerMinute: readPositiveInteger(

"REQUESTS_PER_MINUTE",

30,

1,

10000

),

upstreamTimeoutMs: readPositiveInteger(

"UPSTREAM_TIMEOUT_MS",

30000,

1000,

120000

)

});

For local development, load environment values before starting Node. One straightforward option is to export the values in your shell. Deployment platforms should inject server-side environment variables through their own protected configuration mechanism. The code above intentionally does not rely on an unverified configuration library.

```

Step 3: Build the Node.js Gateway


Create src/server.js. The server is complete and uses only Node.js modules. It accepts POST /api/chat and GET /health. It handles preflight requests for one configured browser origin, but CORS is not authentication. A public gateway still needs an authentication and authorization design appropriate to its users.


import crypto from "node:crypto";
```

import http from "node:http";

import { config } from "./config.js";

const rateWindows = new Map();

function sendJson(response, statusCode, body, requestId) {

const payload = JSON.stringify(body);

response.writeHead(statusCode, {

"Content-Type": "application/json; charset=utf-8",

"Content-Length": Buffer.byteLength(payload),

"X-Request-Id": requestId,

"Cache-Control": "no-store"

});

response.end(payload);

}

function getRequestId(request) {

const supplied = request.headers["x-request-id"];

if (typeof supplied === "string" && supplied.length > 0 && supplied.length <= 128) {

return supplied;

}

return crypto.randomUUID();

}

function applyCors(request, response) {

const origin = request.headers.origin;

if (origin === config.allowedOrigin) {

response.setHeader("Access-Control-Allow-Origin", origin);

response.setHeader("Vary", "Origin");

response.setHeader("Access-Control-Allow-Methods", "POST, GET, OPTIONS");

response.setHeader("Access-Control-Allow-Headers", "Content-Type, X-Request-Id");

response.setHeader("Access-Control-Max-Age", "86400");

}

}

function clientKey(request) {

return request.socket.remoteAddress ?? "unknown";

}

function isRateLimited(request) {

const key = clientKey(request);

const now = Date.now();

const windowStart = now - 60_000;

const recent = (rateWindows.get(key) ?? []).filter((timestamp) => timestamp > windowStart);

if (recent.length >= config.requestsPerMinute) {

rateWindows.set(key, recent);

return true;

}

recent.push(now);

rateWindows.set(key, recent);

return false;

}

function readJsonBody(request) {

return new Promise((resolve, reject) => {

let totalBytes = 0;

const chunks = [];

```
request.on("data", (chunk) => {
  totalBytes += chunk.length;
  if (totalBytes > config.maxBodyBytes) {
    reject(new Error("BODY_TOO_LARGE"));
    request.destroy();
    return;
  }
  chunks.push(chunk);
});

request.on("end", () => {
  try {
    const text = Buffer.concat(chunks).toString("utf8");
    resolve(JSON.parse(text));
  } catch {
    reject(new Error("INVALID_JSON"));
  }
});

request.on("error", reject);
```

});

}

function validateChatRequest(value) {

if (!value || typeof value !== "object" || Array.isArray(value)) {

return "Request body must be a JSON object.";

}

if (!Array.isArray(value.messages) || value.messages.length === 0) {

return "messages must be a non-empty array.";

}

if (value.messages.length > config.maxConversationMessages) {

return `messages must contain at most ${config.maxConversationMessages} items.`

;

}

const roles = new Set(["system", "user", "assistant"]);

let systemCount = 0;

for (const message of value.messages) {

if (!message || typeof message !== "object" || Array.isArray(message)) {

return "Each message must be an object.";

}

if (!roles.has(message.role)) {

return "Each message role must be system, user, or assistant.";

}

if (typeof message.content !== "string" || message.content.trim().length === 0) {

return "Each message content value must be a non-empty string.";

}

if (message.content.length > config.maxMessageChars) {

return `Each message content value must be at most ${config.maxMessageChars} characters.`

;

}

if (message.role === "system") systemCount += 1;

}

if (systemCount > 1) return "Only one system message is allowed.";

if (value.messages[0].role === "assistant") {

return "The first message cannot use the assistant role.";

}

return null;

}

async function forwardToModelService(input, requestId) {

const timeout = AbortSignal.timeout(config.upstreamTimeoutMs);

const upstreamResponse = await fetch(config.modelServiceUrl, {

method: "POST",

headers: {

"Content-Type": "application/json",

"X-Request-Id": requestId

},

body: JSON.stringify({ requestId, messages: input.messages }),

signal: timeout

});

const contentType = upstreamResponse.headers.get("content-type") ?? "";

const responseBody = contentType.includes("application/json")

? await upstreamResponse.json()

: { message: await upstreamResponse.text() };

return { status: upstreamResponse.status, body: responseBody };

}

const server = http.createServer(async (request, response) => {

const requestId = getRequestId(request);

applyCors(request, response);

if (request.method === "OPTIONS") {

response.writeHead(204, { "X-Request-Id": requestId });

response.end();

return;

}

if (request.method === "GET" && request.url === "/health") {

sendJson(response, 200, { status: "ok", service: "nodejs-ai-api-gateway" }, requestId);

return;

}

if (request.method !== "POST" || request.url !== "/api/chat") {

sendJson(response, 404, { error: { code: "NOT_FOUND", message: "Route not found." } }, requestId);

return;

}

if (isRateLimited(request)) {

sendJson(response, 429, { error: { code: "RATE_LIMITED", message: "Try again shortly." } }, requestId);

return;

}

if (!request.headers["content-type"]?.includes("application/json")) {

sendJson(response, 415, { error: { code: "UNSUPPORTED_MEDIA_TYPE", message: "Use application/json." } }, requestId);

return;

}

try {

const input = await readJsonBody(request);

const validationError = validateChatRequest(input);

```
if (validationError) {
  sendJson(response, 400, { error: { code: "INVALID_REQUEST", message: validationError } }, requestId);
  return;
}

const upstream = await forwardToModelService(input, requestId);
sendJson(response, upstream.status, { requestId, data: upstream.body }, requestId);
```

} catch (error) {

if (error.message === "BODY_TOO_LARGE") {

sendJson(response, 413, { error: { code: "BODY_TOO_LARGE", message: "Request body exceeds the configured limit." } }, requestId);

return;

}

```
if (error.message === "INVALID_JSON") {
  sendJson(response, 400, { error: { code: "INVALID_JSON", message: "Request body must contain valid JSON." } }, requestId);
  return;
}

console.error(JSON.stringify({ requestId, error: String(error) }));
sendJson(response, 502, { error: { code: "MODEL_SERVICE_ERROR", message: "The model service could not be reached or did not complete the request." } }, requestId);
```

}

});

server.listen(config.port, () => {

console.log(JSON.stringify({ event: "listening", port: config.port }));

});

function shutdown(signal) {

console.log(JSON.stringify({ event: "shutdown_started", signal }));

server.close(() => process.exit(0));

setTimeout(() => process.exit(1), 10_000).unref();

}

process.on("SIGINT", () => shutdown("SIGINT"));

process.on("SIGTERM", () => shutdown("SIGTERM"));

The local rate limiter uses process memory and the socket address as its key. This is suitable only for a single-process demonstration. When a service runs behind a load balancer or on multiple instances, each process has its own memory and may observe a different client address. Use a shared, deliberately designed rate-limit mechanism and authenticated identity signals before treating limits as an enforceable organisation-wide control.

The server logs only an opaque request ID and an error string in the catch path. It does not intentionally log message content. This is a practical default because prompts may contain sensitive business information, source code, or personal information. If you need content logging for a defined evaluation process, decide separately what may be retained, who may access it, and how deletion and incident response work.

```

Step 4: Run and Test the Gateway


Set the environment variables in your shell, then start the server. The exact shell syntax differs by operating system. On a Unix-like shell, the following starts the application with the example local URL.


export PORT=3001
```

export MODEL_SERVICE_URL=[http://127.0.0.1:8080/generate](http://127.0.0.1:8080/generate)

export ALLOWED_ORIGIN=[http://localhost:3000](http://localhost:3000)

npm run dev

First test the health endpoint. It does not call the model service.

```
curl --include http://localhost:3001/health
```

Then send a valid chat request. This request will reach the configured model service, so it can succeed only when that service is available and implements the contract expected by your application.

```
curl --include --request POST http://localhost:3001/api/chat \
  --header "Content-Type: application/json" \
  --data '{
    "messages": [
      { "role": "system", "content": "Answer concisely." },
      { "role": "user", "content": "What is an API gateway?" }
    ]
  }'
```

Also test negative paths. Send malformed JSON to confirm the gateway returns `400`

. Send a request with an invalid role to verify the application-level schema. Finally, temporarily stop the model service to check the gateway’s `502`

response. Failure testing is essential because network availability and upstream behaviour are not guaranteed.

```
curl --include --request POST http://localhost:3001/api/chat \
  --header "Content-Type: application/json" \
  --data '{"messages":[{"role":"unknown","content":"Hello"}]}'

What to Build Next


Start with authentication. CORS controls which compliant browsers may read a response; it does not establish who is allowed to use an endpoint. Add an authentication mechanism appropriate to your application, then associate requests with a user, service, organisation, or tenant before applying meaningful quotas.


Next, define the model-service contract explicitly. Document the accepted request body, expected response body, timeout behaviour, error statuses, and versioning strategy. If you introduce a queue for long-running work, make the request lifecycle explicit rather than pretending every task will complete during one synchronous HTTP call. The verified context’s example architecture includes a distributed task queue alongside a Node.js backend and model service, which is a useful direction to evaluate for workloads that do not fit a simple request-response interaction.


For scale, put the gateway behind infrastructure that your team operates and monitors. The verified context describes a load balancer as part of an example production stack. If you deploy multiple gateway instances, replace the in-memory limiter with a shared approach, ensure request IDs flow through every service, and test how client address information is handled by your proxy configuration.


For GCC and Middle East deployments, avoid unsupported assumptions about local hosting, legal requirements, or named government programmes. Instead, have legal, security, and data-governance stakeholders assess where data is processed, what information is sent to the model service, how long records are retained, and which regional contractual or regulatory obligations apply to the specific organisation. A gateway boundary makes those controls easier to place consistently, but it does not itself establish compliance.


Finally, keep dependencies and runtime components patched. The verified context includes a National Vulnerability Database entry, reinforcing the practical need to monitor security advisories relevant to the software you operate. Review advisories, test upgrades, and maintain a repeatable deployment process. A compact gateway with a clear contract is easier to test, audit, and evolve than provider-specific calls scattered across browser and backend applications.

```