The naive way to map a Google Doc into a CMS is to send the whole document to an LLM and ask it to figure out which sections go where. That works at small scale, with a small content model. It stops working when your content model has forty content types, each with a dozen fields, and each document has a different structure. You end up burning a large context window on mostly-irrelevant content types, paying for tokens that don't help, and getting worse results because the model is reasoning about the full space of possibilities instead of a curated shortlist.
The pipeline I was working on (a Google Docs → Contentful import) addressed this with a deterministic pre-filtering stage that runs before any LLM call. Its job is not to do the mapping. Its job is to narrow the problem: segment the document into contiguous regions, score each region against every content type, and hand the LLM a small set of (scope, candidate content types) pairs instead of a full Cartesian product. This post is about how that scoring works and why the design choices look the way they do.
The document is treated as a sequence of blocks. Any block that is a tab opener, an H1, or an H2 starts a new scope. Every block between two openers belongs to the scope opened by the first one. The result is a flat array of DocumentScope
objects:
export type ScopeReasonCode = 'tab-label' | 'heading-match' | 'body-term-match'
export type ContentTypeMatch = {
contentTypeId: string
score: number
reasons: ScopeReasonCode[]
}
export type DocumentScope = {
openerText: string | null
openerKind: 'tab' | 'heading' | 'document-root'
blocks: NormalizedBlock[]
matches: ContentTypeMatch[]
topMatch: ContentTypeMatch | null
needsReview: boolean
}
openerText
is the raw text of the tab label or heading that opened the scope. blocks
is the contiguous slice of the document. matches
is every content type scored against this scope in descending score order. topMatch
is a convenience pointer to the first match - null
if there were no matches above the noise floor. needsReview
is a flag, not a gate.
The function signature is straightforward:
export function segmentDocumentScopes(
doc: NormalizedDocument,
slice: ContentModelOntologySlice,
): DocumentScope[]
The ContentModelOntologySlice
is a pre-built graph of content type nodes, field nodes, and the intentTerms
sets derived from each; more on those below.
The score for a (scope, content type) pair is Jaccard similarity over tokenized text:
function jaccardSimilarity(a: Set<string>, b: Set<string>): number {
if (a.size === 0 && b.size === 0) return 0
let intersection = 0
for (const term of a) {
if (b.has(term)) intersection++
}
const union = a.size + b.size - intersection
return union === 0 ? 0 : intersection / union
}
function tokenize(text: string): Set<string> {
return new Set(
text
.toLowerCase()
.replace(/[^a-z0-9\s]/g, ' ')
.split(/\s+/)
.filter((t) => t.length > 1),
)
}
|A ∩ B| / |A ∪ B|
. That is the entire base score. No TF-IDF weights, no vector embeddings, no pre-computed corpus statistics. The score for a scope with { "product", "info", "title" }
against a content type with intent terms { "product", "title", "description" }
is 2 / 4 = 0.5
.
The reason for Jaccard is not that it produces better scores than cosine similarity over TF-IDF vectors. It almost certainly does not. The reason is that this function runs synchronously on every request, before any LLM call. Embeddings require a model call. Cosine similarity over TF-IDF requires pre-computing corpus statistics from the full set of documents, which is not available at request time. Jaccard over a simple tokenized set is O(|A| + |B|), deterministic, reproducible, and has no external dependencies. It does not need to be perfect; it needs to be good enough to narrow forty content types to three, fast enough that it adds no measurable latency, and auditable enough that when it gets something wrong, you can read the code and understand exactly why.
A tab label that reads "Product Info" is a categorically stronger signal than a body paragraph that mentions "product" once. The base Jaccard score treats every term equally regardless of where it appears. The bonus layer addresses this:
function computeScore(
scope: RawScope,
ctNode: ContentTypeNode,
): { score: number; reasons: ScopeReasonCode[] } {
const intentTerms = ctNode.intentTerms
const bodyTerms = tokenize(scope.blocks.map((b) => b.text).join(' '))
const baseScore = jaccardSimilarity(bodyTerms, intentTerms)
const reasons: ScopeReasonCode[] = []
let bonus = 0
if (scope.openerKind === 'tab' && scope.openerText) {
const tabTerms = tokenize(scope.openerText)
const tabOverlap = jaccardSimilarity(tabTerms, intentTerms)
if (tabOverlap > 0) {
bonus += 0.2 * tabOverlap
reasons.push('tab-label')
}
}
if (scope.openerKind === 'heading' && scope.openerText) {
const headingTerms = tokenize(scope.openerText)
const headingOverlap = jaccardSimilarity(headingTerms, intentTerms)
if (headingOverlap > 0) {
bonus += 0.1 * headingOverlap
reasons.push('heading-match')
}
}
if (baseScore > 0) {
reasons.push('body-term-match')
}
return { score: baseScore + bonus, reasons }
}
The bonus is a multiplier on the overlap in the structural position, not a flat constant. A tab label that shares half its terms with the content type's intent terms contributes 0.2 × 0.5 = 0.1
to the score. A tab label that fully matches contributes 0.2
. The multiplier means the bonus scales with how strongly the structural position matched, not just whether it matched at all. A document that opens a tab called "Product Info" against a content type called "Product" with intent terms derived from that name gets a materially higher score than a document that mentions "product" once in a body paragraph with no structural indicator.
The three reason codes: tab-label
, heading-match
, body-term-match
; exist for observability, not routing. When the LLM receives a scope with its top matches, the reason codes tell it (and any downstream monitoring) what evidence the scorer used. A match driven purely by body-term-match
with a low score deserves more skepticism from the LLM than one driven by tab-label
with a high score. Making the evidence explicit, not just the verdict, gives the downstream consumer something to work with.
The ContentModelOntologySlice
is built deterministically from the raw content type definitions before any document is processed:
export function buildContentModelOntologySlice(
contentTypes: ContentType[],
): ContentModelOntologySlice {
const ctNodes: ContentTypeNode[] = contentTypes.map((ct) => {
const terms = new Set<string>()
// CT-level signals
for (const t of tokenize(ct.sys.id)) terms.add(t)
for (const t of tokenize(ct.name)) terms.add(t)
if (ct.description) for (const t of tokenize(ct.description)) terms.add(t)
// Field-level signals
for (const field of ct.fields) {
for (const t of tokenize(field.id)) terms.add(t)
for (const t of tokenize(field.name)) terms.add(t)
if (field.type === 'Link' && field.validations) {
for (const v of field.validations) {
if (v.linkContentType) {
for (const linkedCtId of v.linkContentType) {
for (const t of tokenize(linkedCtId)) terms.add(t)
}
}
}
}
// helpText, validation summaries, etc.
}
return { contentTypeId: ct.sys.id, intentTerms: terms, fields: [] }
})
return { ctNodes, fieldNodes: [], edges: [] }
}
The key decision here is tokenizing field IDs, content type IDs, and validation rules, not just display names and descriptions. A content type named ctProductVariant
with a field named linkedEntries
that has a linkContentType: ["BlogPost"]
validation produces intent terms including ctproductvariant
, linkedentries
, and blogpost
. A content type with a sparse or technical name (pgBlogEntry
) still has vocabulary the scorer can match against; it isn't limited to whatever its display name happens to be.
This matters because many production content models have programmatic IDs that carry semantic information. A content type called "Page: Blog Entry" in the display name but pgBlogEntry
in the API name will produce terms for both. The validation relationships (linkContentType
, allowedContentTypes
) bring in referenced content type IDs, which means a scope that references "blog posts" in passing can score against a container content type that links to BlogPost
, even if the container's own name doesn't mention it.
After scoring, three conditions set needsReview: true
on a scope:
function computeNeedsReview(scope: RawScope, matches: ContentTypeMatch[]): boolean {
// No content blocks — nothing to score against
if (scope.blocks.length === 0) return true
const top = matches[0]
const second = matches[1]
// Weak top score — no content type matched confidently
if (!top || top.score < 0.1) return true
// Ambiguous — top two are nearly tied
if (second && top.score - second.score < 0.05) return true
return false
}
None of these conditions throw an error or halt the pipeline. The pipeline continues with all scopes, flagged or not. The needsReview
flag is a signal passed downstream to the LLM (and to monitoring), not a gate.
The reason for this design is that hard-failing on low scores would block legitimate edge cases: a document with no headings produces a single large scope that scores diffusely across many content types; a content type with sparse intent terms produces low scores even against obviously relevant documents. These are not pipeline errors; they are cases where the deterministic scorer ran out of signal. The right behavior is to continue and let the LLM handle those scopes with the full context of knowing the scorer wasn't confident, not to abort the import and force a manual retry.
The top-2 gap check deserves its own explanation. A score of 0.15 vs. 0.14 is not a decision, it is noise in a simple set-overlap function. Rather than let a near-tie pass through as a confident first-place assignment, the gap check flags it explicitly. The LLM sees two near-equal candidates and needsReview: true
, which is accurate: the scorer does not have enough signal to pick a winner, and the LLM should treat both candidates seriously rather than assuming the top-ranked one is correct.
The specific implementation here, Jaccard similarity, structural bonuses, needsReview
flags, is tied to this pipeline. The underlying decisions are not:
needsReview
as a first-class output is better than crashing or silently passing through low-confidence decisions.The LLM in this pipeline gets a scoped problem instead of a full document-by-content-model matrix. That is the only job of this stage. Getting it right deterministically means the expensive, slow, creative part of the pipeline; the LLM spends its context on decisions it actually needs to make.