Build a Secure Node.js AI API Gateway

What You Are Building

This tutorial builds a deliberately small Node.js AI API gateway. A client sends a JSON request to POST /api/chat. The gateway checks that the request has the expected shape, rejects oversized payloads, assigns a request ID, applies a local rate limit, and forwards the validated request to a separate model service over HTTP. The gateway then returns the model service response to the caller.

The architectural idea is grounded in a commonly used separation of concerns: a web backend serves HTTP APIs while a model service performs the core predictive task. The verified context describes an example stack with a web frontend, load balancer, Node.js REST API backend, distributed task queue, and a model service. This article implements only the Node.js gateway boundary. It does not claim to prescribe a complete production architecture, a particular model vendor, or a particular model-serving framework.

Keeping the gateway separate from the model service gives an application one place to apply input rules and operational controls before work reaches an AI system. It also means browser applications call your service rather than receiving a sensitive upstream credential. The exact authentication, authorization, data-retention, safety review, and deployment requirements depend on your organisation and use case, so they are intentionally not represented as universal defaults in this example.

The implementation avoids unverified provider-specific SDK calls, model names, token limits, and model-response schemas. Instead, it defines a small JSON contract owned by this application. That is useful when the model service is an internal service, a separately operated inference API, or an adapter that you maintain elsewhere.

Gateway Responsibilities and Boundaries

A gateway should have a narrow job. In this tutorial it does five things: it accepts HTTP requests, parses limited-size JSON, validates required fields, records an opaque request ID, and forwards safe input to an upstream model service. It does not attempt to decide whether a generated answer is correct. It does not train a model. It does not embed keys in a frontend bundle. It also does not assume that every model service uses the same request or response structure.

Our public request contract is intentionally simple:

{
  "messages": [
    { "role": "user", "content": "Explain an API gateway." }
  ]
}

The gateway accepts up to 20 messages, permits only system, user, and assistant roles, and limits each message to 12,000 characters by default. These are application controls rather than token measurements. Characters and model tokens are not the same unit, so a character limit should not be presented as a precise cost or usage limit.

The upstream service in this tutorial receives a wrapper containing a request ID and the validated messages. Its expected successful response is JSON. The gateway does not transform that JSON into a vendor-neutral completion format because no provider response format is verified in the available context. Owning a small, documented internal contract is safer than guessing at a provider API.

Step 1: Create the Project and Environment File

Create a new project directory. This version uses no third-party dependency, which makes the example easy to inspect and keeps its runtime surface small. The built-in Node.js HTTP server is sufficient for a focused gateway demonstration.

mkdir nodejs-ai-api-gateway
cd nodejs-ai-api-gateway
npm init -y
npm pkg set type=module
npm pkg set scripts.start="node src/server.js"
npm pkg set scripts.dev="node --watch src/server.js"
npm pkg set engines.node=">=18.0.0"
mkdir -p src

Create a .gitignore file before creating local configuration. Never commit credentials or deployment-specific environment files to source control.

node_modules
.env
.env.local
.env.production
logs
coverage
npm-debug.log*
.DS_Store

Next, create .env. MODEL_SERVICE_URL is the only required upstream setting. The URL must point to a service that your deployment can reach and is authorised to use. The example uses a loopback URL only as a local-development value.

PORT=3001
MODEL_SERVICE_URL=http://127.0.0.1:8080/generate
ALLOWED_ORIGIN=http://localhost:3000
MAX_BODY_BYTES=262144
MAX_MESSAGE_CHARS=12000
MAX_CONVERSATION_MESSAGES=20
REQUESTS_PER_MINUTE=30
UPSTREAM_TIMEOUT_MS=30000

Do not put upstream credentials in browser-exposed environment variables. If the model service requires credentials, keep them in the gateway’s server-side deployment environment and attach them only on the server. This tutorial does not include an authorization header because its name, format, and credential lifecycle are not established by the verified context.

Step 2: Validate Configuration at Startup

Create src/config.js. Environment variables arrive as strings, so the module explicitly parses numerical values and fails early when required configuration is invalid. Startup validation is preferable to discovering a malformed port or URL only after traffic arrives.

import process from "node:process";function readPositiveInteger(name, fallback, minimum, maximum) {
const raw = process.env[name] ?? String(fallback);
const value = Number.parseInt(raw, 10);if (!Number.isInteger(value) || value < minimum || value > maximum) {
throw new Error(`${name} must be an integer from ${minimum} to ${maximum}.`);
}return value;
}function readUrl(name) {
const raw = process.env[name];if (!raw) {
throw new Error(`${name} is required.`);
}try {
return new URL(raw).toString();
} catch {
throw new Error(`${name} must be a valid URL.`);
}
}export const config = Object.freeze({
port: readPositiveInteger("PORT", 3001, 1, 65535),
modelServiceUrl: readUrl("MODEL_SERVICE_URL"),
allowedOrigin: process.env.ALLOWED_ORIGIN ?? "http://localhost:3000",
maxBodyBytes: readPositiveInteger("MAX_BODY_BYTES", 262144, 1024, 1048576),
maxMessageChars: readPositiveInteger("MAX_MESSAGE_CHARS", 12000, 1, 100000),
maxConversationMessages: readPositiveInteger(
"MAX_CONVERSATION_MESSAGES",
20,
1,
100
),
requestsPerMinute: readPositiveInteger(
"REQUESTS_PER_MINUTE",
30,
1,
10000
),
upstreamTimeoutMs: readPositiveInteger(
"UPSTREAM_TIMEOUT_MS",
30000,
1000,
...
Continue Reading
Log in for free to read the rest of this article and access exclusive AI tools.
Log in / Register