{"slug": "my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math", "title": "My AI Content Got Flagged as Templated. The Fix Was N-gram Math.", "summary": "A developer whose AI-generated comparison articles were silently penalized by Google discovered the cause was structural fingerprinting, not plagiarism. By analyzing n-gram Jaccard similarity across their corpus, they found that templated paragraph slots created near-duplicate sentence structures. The fix involved splitting paragraphs into short, independently varied slots selected via hashing, yielding thousands of distinct combinations and measurably distinct articles.", "body_md": "Google doesn't read your articles. It fingerprints them.\n\nEarlier this year, a batch of comparison articles across my B2B sites got hit with the quietest penalty in SEO: nothing at all. Indexed fine, rendered fine, ranked nowhere. Average position 76 across five domains, three months, ~50,000 impressions, seven clicks.\n\nWhen I finally stopped looking at rankings and started looking at my own content, I found something embarrassing. My AI-generated articles shared sentence-level fingerprints with each other. Not plagiarism — every article passed every plagiarism checker as \"unique.\" But structurally, hundreds of them were the same article wearing different words.\n\nThis is how I found the fingerprints, reverse-engineered the math that detects them, and rebuilt the generation pipeline until every pair of articles was measurably distinct. Working code included.\n\nDuplicate detection at scale doesn't compare whole documents. It compares **n-grams** — sliding windows of n consecutive words — usually as a \"shingle\" set. Two documents are \"near-duplicates\" when their shingle sets overlap heavily, measured by Jaccard similarity:\n\n```\nJaccard(A, B) = |A ∩ B| / |A ∪ B|\n```\n\nAn 8-gram is a good window for sentence-level work: long enough that a shared 8-word sequence is almost never coincidence, short enough that it catches partial rewrites. If two paragraphs share most of their 8-grams, they're the same paragraph for ranking purposes — no matter that the words around them differ.\n\nHere's the entire detector, which is the point — the math is simple enough that there's no excuse for not running it on your own content:\n\n``` php\nimport re\nfrom pathlib import Path\n\ndef ngrams(text: str, n: int = 8) -> set[str]:\n    words = re.sub(r\"[^\\w ]\", \"\", text.lower()).split()\n    return {\" \".join(words[i : i + n]) for i in range(len(words) - n + 1)}\n\ndef jaccard(a: set, b: set) -> float:\n    return len(a & b) / len(a | b) if (a | b) else 0.0\n```\n\nRun it across every pair of same-position paragraphs in a content batch, and you get a heatmap of how templated your corpus actually is.\n\nI write \"X vs Y\" comparison articles. The generation pipeline had a FAQ block, a verdict block, and a few recurring analysis blocks — each with a handful of pre-written variants the pipeline rotated through.\n\nThe scan results, across ~300 articles on three sites:\n\nEvery article was \"unique.\" The corpus was a Xerox machine.\n\nThe obvious fix: write more variants per block, rotate them. I went from 3 variants to 6.\n\nThe scan barely moved. Two lessons fell out of the data:\n\n**Long slots dominate the fingerprint.** One block had a single 20-word sentence slot with 6 variants. But a 20-word slot contributes 13 of its own 8-grams — roughly 60–70% of the paragraph's entire shingle set. With 6 variants across 70+ articles, dozens of pairs shared a variant and collided on most of their fingerprints. Six options, seventy articles, one shared long slot: the math was doomed before I wrote a single new variant.\n\n**Rotation creates only `k` distinct skeletons.** `article_index % k` means with k variants, you have exactly k distinct articles, repeated forever. Jaccard between two articles sharing a variant: high. Guaranteed.\n\nThe winning design flips the unit of variation. Instead of varying whole paragraphs, split each paragraph into **3–4 short slots** (each ≤ 12 words), give each slot **3–6 independent phrasings**, and select per-article with a hash:\n\n``` python\nimport hashlib\n\ndef pick(options: list[str], article_slug: str, slot_name: str, reseed: int = 0) -> str:\n    key = f\"{article_slug}:{slot_name}:{reseed}\".encode()\n    h = int(hashlib.md5(key).hexdigest(), 16)\n    return options[h % len(options)]\n```\n\nThe combinatorics do the rest. A 4-slot paragraph with 3–6 options per slot yields **216–1,296 distinct combinations**. For 70 articles, the space is 3–18x larger than the demand — plenty of room for every article to be structurally different from every other.\n\nRule of thumb I'd now tattoo on the inside of every content pipeline: **the slot-combination space must exceed the article count with margin, or you've built a rotation, not a generator.**\n\nPer-slot independence isn't quite enough. Two articles can still collide by unlucky draws — same slot options in the same order. So generation became a constraint problem: assign every article a combination such that **every pair stays under a Jaccard threshold** (I targeted < 0.5, with < 0.3 as the comfort zone).\n\nMy first approach was greedy: generate all articles, find conflicting pairs, re-seed the loser, repeat. It oscillated forever — the conflict count bounced between 5 and 10 for over a thousand reseed iterations. Classic local-search thrash: fixing one pair broke another.\n\nWhat converged was embarrassingly simple: **sequential construction.**\n\n``` php\ndef resolve(articles: list, slots: dict, max_j: float = 0.5) -> dict:\n    chosen = {}          # slug -> (fingerprint set, combo)\n    for slug in articles:\n        for reseed in range(200):          # per-article attempts\n            combo = [pick(slots[s], slug, s, reseed) for s in slot_names]\n            fp = ngrams(\" \".join(combo))\n            if all(jaccard(fp, chosen[o][0]) < max_j for o in chosen):\n                chosen[slug] = (fp, combo)\n                break\n        else:\n            raise RuntimeError(f\"no valid combo for {slug}\")\n    return chosen\n```\n\nPlace articles one at a time. Each new article checks itself against the already-placed set only, and once placed, never moves. No oscillation is possible because nothing ever gets un-fixed. Deterministic, fast, and it converged on the first pass for every site.\n\nGreedy restart kept thrashing because it re-opened solved articles. Sequential construction works because it never does.\n\nAfter rebuilding all three sites' recurring blocks on short-slot combinations:\n\n| Metric | Before | After | \n|---|---|---|\n| Paragraph pairs at J ≥ 0.5 | dozens | **0** | \n| Global max pairwise Jaccard | 1.00 | **0.48** | \n| Verbatim-duplicate paragraphs | hundreds | **0** | \n\nZero pairs above the near-duplicate threshold. The remaining sub-0.5 pairs are legitimately different sentences sharing an 8-gram or two — exactly what honest topically-similar content looks like.\n\nThe deeper lesson cuts against how most AI content tools work: a template with slots isn't a feature, it's a liability. The pipeline that replaced all this doesn't have fixed blocks at all — it reads the live SERP for each keyword and drafts from that pattern, so every article's structure is derived from what actually ranks for that query, not from a shared skeleton. That tool is [SerpCraft](https://serpcraft.io) — free tier, no card, and the SERP analysis page works even if you never let it write a word.\n\nIf you've run n-gram analysis on your own AI content, I'd like to hear how bad it was. It was worse than I expected, every time I looked.", "url": "https://wpnews.pro/news/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math", "canonical_source": "https://dev.to/toolkitcreators/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math-1fei", "published_at": "2026-09-07 13:00:33+00:00", "updated_at": "2026-09-07 13:29:12.633077+00:00", "lang": "en", "topics": ["artificial-intelligence", "natural-language-processing", "developer-tools"], "entities": ["Google"], "alternates": {"html": "https://wpnews.pro/news/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math", "markdown": "https://wpnews.pro/news/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math.md", "text": "https://wpnews.pro/news/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math.txt", "jsonld": "https://wpnews.pro/news/my-ai-content-got-flagged-as-templated-the-fix-was-n-gram-math.jsonld"}}