# Implementing Node.js Support Triage — Summarize PDF Pages with Embeddings

> Source: <https://dev.to/peterparker8991/implementing-nodejs-support-triage-summarize-pdf-pages-with-embeddings-20j9>
> Published: 2026-08-15 02:13:56+00:00

Short answer: for B2B support triage, split PDF retrieval from answer generation, rerank a small evidence set, and let the final summary assign a queue only when the cited pages support that decision.

| Design | Triage quality | Request latency | Operational load | Use it when |
|---|---|---|---|---|
| Embedding search, then summary | Baseline | Lowest of these three | Low | The corpus is narrow and ticket language closely matches the manuals |
| Embedding search, rerank, then summary | Better evidence ordering | One extra model step | Medium | Wrong routing costs more than one extra call |
| Precomputed issue-to-page rules | Predictable for known cases | Low | High maintenance | The issue taxonomy is stable and auditability outranks recall |

The practical default is the middle row. Keep the first row as a latency fallback, and keep deterministic rules for a small set of high-consequence queues. This isn't a model contest. It is a revenue-per-hour decision: spend complexity where a bad handoff creates another support cycle, and outsource the undifferentiated model calls behind three tiny interfaces.

Ship the narrow path first.

A ticket such as “CSV export drops custom fields after the workspace migration” may share very few words with the page that explains schema mapping. Embedding search is useful because it can retrieve semantically related pages, but retrieval rank is still only a candidate order. The summarizer should not inherit that order as truth. Reranking gives the ticket and candidate pages a second comparison step before generation, which is the part worth paying for when queue selection depends on a precise product condition.

For a one-person SaaS, “quality” needs an operational definition. I use an evidence contract rather than a vague score: the output names one queue, cites page identifiers, states what the evidence supports, and returns `needs_review`

when the pages do not justify a route. That contract can be checked without pretending that fluent prose is correct. It also keeps weekly shipping possible; a new manual changes indexed content, not the orchestration code.

Measure that.

Latency has two clocks. The customer sees time to acknowledgement, while the support operator sees time to a trustworthy route. A fast acknowledgement can be sent before enrichment finishes, so the internal triage path does not need to win an artificial race against the first response. Still, cap each stage. Retrieval, reranking, and summarization should have independent timeouts because a single end-to-end timeout won't tell you which budget was exhausted.

The catch is that reranking every ticket is wasteful when the first retrieval result is already decisive. Use a policy, not instinct: skip the extra pass only for an exact document or error-code match that your own evaluation set has shown to be reliable. I’m not sure a universal similarity threshold exists across embedding models and corpora; a labeled ticket set from the actual help desk is what resolves that uncertainty.

Treat a PDF page as evidence with identity, not as an anonymous string. Preserve the document ID, page number, revision, and text from extraction onward. Then embed the ticket query, retrieve more pages than the final prompt can afford, rerank those candidates against the full ticket, and pass only the best supported pages to the summary step. The final result must carry the selected page IDs back out.

That sequence prevents a common architecture error: using the generated summary to reconstruct citations after the fact. Once page identity has been discarded, matching prose back to a source is another fuzzy retrieval problem. Keep the IDs attached instead. Simple.

The boundaries matter more than a clever framework. Define three ports: vector search, reranking, and structured generation. Each adapter can call a hosted API, an internal service, or a self-hosted model; the pipeline does not care. It does care about input order, bounded candidate counts, and whether every returned citation belongs to the supplied evidence.

The quality gate should be built from representative support tickets, including vague wording, copied error text, stale terminology, and two issues described in one message. Label the acceptable queues and supporting pages. Then compare the embedding-only path with the reranked path on the same cases. Your mileage may vary because manuals, ticket vocabulary, and queue taxonomies differ; publish no generic accuracy claim when the relevant evidence is local.

The following code is the orchestration layer. It deliberately owns the request and response contracts while leaving model transport in injected adapters. That keeps the example runnable in tests without baking a commercial endpoint into the application.

```
type PdfPage = {
  id: string;
  documentId: string;
  revision: string;
  pageNumber: number;
  text: string;
};

type Ticket = {
  id: string;
  subject: string;
  body: string;
};

type RetrievedPage = PdfPage & { similarity: number };
type RankedPage = RetrievedPage & { rerankScore: number };

type TriageResult = {
  queue: "billing" | "data" | "integrations" | "needs_review";
  summary: string;
  citedPageIds: string[];
  confidence: "high" | "medium" | "low";
};

interface SemanticIndex {
  search(query: string, limit: number, signal: AbortSignal): Promise<RetrievedPage[]>;
}

interface PageReranker {
  rank(query: string, pages: RetrievedPage[], signal: AbortSignal): Promise<RankedPage[]>;
}

interface StructuredSummarizer {
  summarize(ticket: Ticket, pages: RankedPage[], signal: AbortSignal): Promise<TriageResult>;
}

type PipelinePorts = {
  index: SemanticIndex;
  reranker: PageReranker;
  summarizer: StructuredSummarizer;
};

function ticketQuery(ticket: Ticket): string {
  return `${ticket.subject}\n\n${ticket.body}`.trim();
}

function withDeadline(milliseconds: number): AbortSignal {
  return AbortSignal.timeout(milliseconds);
}

function validateCitations(result: TriageResult, evidence: RankedPage[]): TriageResult {
  const allowed = new Set(evidence.map((page) => page.id));
  const citationsAreValid =
    result.citedPageIds.length > 0 &&
    result.citedPageIds.every((pageId) => allowed.has(pageId));

  if (citationsAreValid) return result;

  return {
    queue: "needs_review",
    summary: "The selected pages do not support an automatic route.",
    citedPageIds: [],
    confidence: "low",
  };
}

export async function triageTicket(
  ticket: Ticket,
  ports: PipelinePorts,
): Promise<TriageResult> {
  const query = ticketQuery(ticket);
  const candidates = await ports.index.search(query, 18, withDeadline(1_500));

  if (candidates.length === 0) {
    return {
      queue: "needs_review",
      summary: "No relevant manual pages were found.",
      citedPageIds: [],
      confidence: "low",
    };
  }

  const ranked = await ports.reranker.rank(
    query,
    candidates,
    withDeadline(1_500),
  );
  const evidence = ranked.slice(0, 6);
  const draft = await ports.summarizer.summarize(
    ticket,
    evidence,
    withDeadline(3_000),
  );

  return validateCitations(draft, evidence);
}
```

The numbers here are configuration choices, not benchmark results: retrieve 18 candidates, retain 6, and allocate separate 1.5-second and 3-second deadlines. Put them in configuration before deployment. A small corpus may need fewer candidates; a manual with repetitive boilerplate may need more. Change one number at a time against the labeled ticket set so a latency win cannot quietly erase the page that supports the correct queue.

Test the orchestration with fixed adapters. This test verifies the failure policy that matters most: a generated citation outside the selected page set forces human review.

``` python
import assert from "node:assert/strict";

const page: RetrievedPage = {
  id: "manual-r7-p42",
  documentId: "admin-manual",
  revision: "r7",
  pageNumber: 42,
  text: "Workspace migrations preserve mapped fields during CSV export.",
  similarity: 0.82,
};

const result = await triageTicket(
  {
    id: "ticket-1042",
    subject: "CSV export after migration",
    body: "Custom fields are missing from the export.",
  },
  {
    index: {
      search: async () => [page],
    },
    reranker: {
      rank: async (_query, pages) => pages.map((item) => ({
        ...item,
        rerankScore: 0.91,
      })),
    },
    summarizer: {
      summarize: async () => ({
        queue: "data",
        summary: "Review the field mapping used by the migrated workspace.",
        citedPageIds: ["unknown-page"],
        confidence: "high",
      }),
    },
  },
);

assert.equal(result.queue, "needs_review");
assert.deepEqual(result.citedPageIds, []);
```

The summarizer adapter should request structured data matching `TriageResult`

, and its prompt should delimit manual pages as untrusted evidence. Do not let instructions found inside a PDF redefine the task or the output schema. OWASP’s LLM application guidance treats prompt injection and sensitive-information disclosure as application risks, so the boundary belongs in the design even when all documents came from your own company. Validate the queue against the application’s allowlist, validate every page ID against the exact evidence slice, and route the ticket to review when either check fails; generation is allowed to abstain, but it is never allowed to invent a new operational destination.

Do not log raw ticket bodies, page text, or generated summaries by default. Log stage duration, candidate count, selected page IDs, document revisions, route outcome, and a correlation ID. Support tickets can contain names, email addresses, contracts, or credentials; GDPR principles include data minimization, so collection and retention need a stated purpose rather than “we might debug it later.”

Stick with embedding search followed directly by summarization when a wrong route is cheap to correct, the manuals are small and distinct, and the measured gain from reranking does not justify its added latency and failure surface. This is the right runner-up for an early product whose operator reviews every suggestion anyway. It also reduces the number of model-dependent components that a solo maintainer has to observe during a weekly release cycle.

Use deterministic issue-to-page rules instead when queue assignment has a stable, explicit trigger: a documented error code, a contract plan identifier, or a known migration version. Rules are awkward for fuzzy language and expensive when taxonomy changes, but their decisions are inspectable. They can coexist with semantic retrieval: run the narrow rules first, then send unmatched tickets through the two-pass pipeline.

Reranking is not suitable when the extra processing would violate the support workflow’s latency budget, when there is no labeled evaluation set to prove it changes decisions, or when compliance policy prohibits sending retrieved text to the available reranker. In those cases, choose the embedding-only path or a self-controlled deterministic path. A polished summary is not compensation for an architecture that cannot meet its operating constraints.

Before release, replay the labeled set, inspect every `needs_review`

transition, and verify that cited page revisions still exist. After release, sample outcomes by queue and document revision rather than reading every ticket payload. The deployment gate is straightforward: ship only when the candidate pages, final citations, and assigned queue remain traceable through the same correlation ID.

That is enough machinery.