# Building a Vertical Corpus Builder: Clean JSONL Datasets for LLM Fine-Tuning > Source: > Published: 2026-08-19 11:11:46+00:00 Raw web pages are terrible training data. Nav bars, cookie banners, "related articles" and ads drown the signal, and near-identical syndicated text pollutes the corpus. If you're fine-tuning a domain model — legal reasoning, medical QA, financial analysis — you want clean vertical text with provenance, not a pile of HTML. I built an actor that goes from seed URLs to a token-aware JSONL dataset in one run. ``` seed URLs → crawl (same-domain BFS) → clean (boilerplate strip) → dedupe (exact + near) → token-aware chunk → JSONL with provenance ``` Three steps matter most: `script` /`style` /`nav` /`footer` /`header` /`aside` and utility blocks (breadcrumbs, shares, comments, menus). One detail that earned its keep: paragraphs where more than half the links are anchor text get dropped too — those are link farms, not content.Every record carries `source` (hostname), `url` , `domain` (your vertical), `title` , and `chunk_index` — so you can filter, cite, or re-weight the corpus later. Not every legal text source works from a datacenter IP. CourtListener sits behind an AWS WAF JavaScript challenge, and Justia 403s datacenter egress — both unusable as plain-HTTP seeds. **Cornell LII** (`law.cornell.edu/supremecourt/text/…` ) serves full opinions as clean HTML and works beautifully. The honest move was documenting that in the README and defaulting the examples to LII rather than pretending every source is reachable. Three LII Supreme Court opinions (Dobbs, Bruen, WV v EPA): `domain=legal` , source `law.cornell.edu` , 0 duplicate texts`output.jsonl` — 443 lines / ~811 KB, plus a `SUMMARY` record: 443 chunks, 2 pages, ~173,000 estimated tokens, no failed sources`maxChunksPerPage` .👉 **Vertical Corpora Builder on Apify Store** While you're here, these might be worth a read: