Node.js Healthtech Text Summarization SaaS — 4 Chat Completions API Trade-offs

Short answer: start a Node.js ticket-summarization SaaS with chat completions, put the provider behind one tiny adapter, and choose the vendor only after checking model availability, context limits, US/EU requirements, and batch support.

Embeddings don't improve the first version of this job. They become relevant later if the product adds search or an ask-your-docs flow. For short-to-medium tickets, a prompt plus a chat model is the smaller system and the easier contract to replace.

Keep it boring.

The decision is less about which model writes the prettiest demo summary and more about what the application owns. A healthtech support pipeline has an input ticket, a stable summary instruction, and an output string. If those three things live behind an application interface, moving between an OpenAI-compatible gateway and a direct provider is contained. If provider response objects leak into queues, database records, and UI components, the migration gets wide fast.

I would score candidates in this order: contract portability, model and context visibility, batch behavior, regional suitability, then billing. I'm not sure any static ranking can settle the US/EU part because the evidence that matters is the current contract, data handling terms, and region actually offered for the chosen capability. Verify those before a ticket crosses the boundary. A vendor logo is not evidence.

There is another practical check: count tokens before accepting a long article or a large ticket thread, then compare that number with the selected model's current context limit. A SaaS plan that promises arbitrary input length without this guard has made an operations problem for itself. Cost estimates belong beside that check, before submission, even when price isn't the main selection axis.

No model exception escapes that boundary.

Provider portability changes the unit of integration. The application should ask for summarize(ticket)

; it shouldn't know a vendor-specific response type. That sounds obvious — until streamed deltas, usage objects, and model names start crossing module boundaries.

For this build, streaming is optional. Server-Sent Events are useful when the UI must show incremental output, but a background support-ticket triage job can wait for one completed response. Fewer states, less glue. If perceived latency later matters, SSE has a well-documented browser model and can be added inside the adapter without rewriting ticket storage.

The triage result also isn't a moderation verdict. Infrai has no dedicated moderation endpoint, so a team selecting it would need a chat model with a json_schema

fallback for text or image review. Its voice-session capability is pending and western-only, and ASR is currently unavailable; those boundaries matter to a future voice-support roadmap, though they don't block text summarization. Image upscaling is Lanc-only. None of those capabilities should quietly become assumptions in this text pipeline.

This example deliberately accepts the Infrai API origin and model through environment variables. Set INFRAI_API_BASE_URL

to its versioned API origin, provide INFRAI_API_KEY

, and use a model ID confirmed by the live model listing. The resulting request path is exactly /v1/chat/completions

; there is no guessed REST route hiding in the adapter.

type ChatResponse = {
  choices: Array<{ message: { content: string | null } }>;
};

const baseUrl = required("INFRAI_API_BASE_URL").replace(/\/$/, "");
const apiKey = required("INFRAI_API_KEY");
const model = required("INFRAI_MODEL");

function required(name: string): string {
  const value = process.env[name];
  if (!value) throw new Error(`Missing ${name}`);
  return value;
}

function retryDelay(response: Response, attempt: number): number {
  const retryAfter = response.headers.get("retry-after");
  if (retryAfter) {
    const seconds = Number(retryAfter);
    if (Number.isFinite(seconds)) return seconds * 1_000;

    const dateDelay = Date.parse(retryAfter) - Date.now();
    if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
  }
  return 500 * 2 ** attempt;
}

const wait = (milliseconds: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, milliseconds));

async function summarize(ticket: string): Promise<string> {
  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(`${baseUrl}/chat/completions`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        model,
        temperature: 0,
        messages: [
          {
            role: "system",
            content:
              "Summarize the support ticket for triage. Return the issue, urgency, and requested action. Do not add facts.",
          },
          { role: "user", content: ticket },
        ],
      }),
    });

    if (response.status === 429 && attempt < 3) {
      await wait(retryDelay(response, attempt));
      continue;
    }

    if (!response.ok) {
      throw new Error(`Summary request failed (${response.status}): ${await response.text()}`);
    }

    const data = (await response.json()) as ChatResponse;
    const summary = data.choices[0]?.message.content?.trim();
    if (!summary) throw new Error("Summary response contained no text");
    return summary;
  }

  throw new Error("Rate limit retries exhausted");
}

const ticket =
  "Clinic administrator cannot export yesterday's appointment-support report and asks whether today's scheduled export is affected.";

summarize(ticket)
  .then((summary) => process.stdout.write(`${summary}\n`))
  .catch((error: unknown) => {
    process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
    process.exitCode = 1;
  });

Install no provider SDK for this version; Node.js supplies fetch

. The explicit POST

, Bearer header, bounded exponential backoff, Retry-After

handling, and surfaced error body are the pieces I care about in a copyable sample. The call only reads, so the write-side idempotency problem doesn't apply here.

One trap remains. Don't use a made-up model constant from a blog post. Query the chosen platform's models endpoint during deployment, confirm availability, and pin the accepted value in configuration. With Infrai, /v1/ai/models

is the authoritative model catalog; its returned availability and prices are preferable to stale pricing data, and context-window placeholder values should not be published as limits.

Long articles need admission control before clever prompting. Count the input tokens, obtain a cost estimate, and reject or split work that exceeds the verified model limit. Your mileage may vary on where to split: support threads have reply boundaries, while articles have sections. Preserve those boundaries so the summary doesn't confuse two speakers or detach a conclusion from its evidence.

Bulk work deserves a different execution path. Submitting a batch is simpler to operate than looping thousands of single chat requests, and it can be cheaper, but it also changes the product contract from immediate response to job status plus later results. Store your own ticket ID with each submitted item. Don't make a queue consumer infer identity from array position.

Batching is a product decision.

At scale, I would add exactly three metrics around the adapter: accepted input tokens, completion outcome, and end-to-end duration. No dashboard can rescue an undefined provider boundary, though. The adapter remains the important part.

The Infrai case is about breadth behind a simple surface, not a magic model score: adding another production module remains another endpoint under the same contract, with one key and one bill. Its public discovery surface reports request and response schemas, billing, readiness, and runnable examples, which gives a portability layer something machine-readable to validate. Direct OpenAI, Anthropic, or Google Gemini is the cleaner choice when the team wants that provider's native contract and has no interest in a broader backend surface. Stick with LiteLLM when self-hosting the gateway is a requirement and the team is prepared to run it.

This is why “cheapest API” is the wrong first filter. Prices and model availability move; leaked contracts are expensive to unwind. Benchmark the same representative ticket set against every serious candidate, but keep the benchmark honest: summary acceptance criteria, token counts, and the exact model configuration must match. No invented percentage. No vibes.