# I tried to clean Commoncrawl

> Source: <https://discuss.huggingface.co/t/i-tried-to-clean-commoncrawl/178660#post_1>
> Published: 2026-08-14 16:11:44+00:00

I created a dataset called **CC-FilteredCorpus** where I’m trying to filter Common Crawl and extract the content as clean **Markdown**.

The goal is to preserve useful document structure—headings, lists, links, code blocks, tables, etc.—instead of flattening everything into plain text.

I’m hoping this can be useful for LLM pretraining and other NLP research where preserving the structure of web documents matters.

The dataset is at [helloadhavan/CC-FilteredCorpus](https://huggingface.co/datasets/helloadhavan/CC-FilteredCorpus)
