Back to Gap Map Model components / Training & synthetic datasets

C4

open / Overall score: 4.0(strong)

Cleaned version of Common Crawl created for the T5 paper and hosted by AI2. It ships several variants, including a cleaned English split of about 305GB and the multilingual mC4 covering 108 languages at roughly 9.7TB.

Openness

5 high confidence

5.0

license: ODC-BY
ungated: yes
dataset_card: yes

Permissively licensed (ODC-BY) and ungated with a full dataset card.

https://huggingface.co/api/datasets/allenai/c4 recorded 2026-09-29
Hub API record lists "task_categories:text-generation" and "license:odc-by" tags for the dataset, plus a top-level gated flag of false.
https://huggingface.co/datasets/allenai/c4 recorded 2026-09-29
The HF dataset page renders the card, with a Dataset Summary saying "A colossal, cleaned version of Common Crawl" and the license metadata odc-by.
https://huggingface.co/datasets/allenai/c4/raw/main/README.md recorded 2026-09-29
The raw dataset card states "We are releasing this dataset under the terms of [ODC-BY]" and describes "A colossal, cleaned version of Common Crawl" in its Dataset Summary section.

Adoption

5 high confidence

5.0

Adoption is measured by Hugging Face downloads of allenai/c4, the only usage channel on record for C4.

https://huggingface.co/api/datasets/allenai/c4 recorded 2026-08-13
1,414,303 downloads in the trailing 30 days for allenai/c4

Capability

3 high confidence

3.0

C4 trained T5. FineWeb and DCLM have since beaten it in head-to-head ablations, so it now serves as a superseded baseline rather than a current choice.

https://arxiv.org/html/2406.11794 recorded 2026-09-29
DCLM paper says "the dataset ranking remains largely stable" across hyperparameter settings, with DCLM-baseline ahead of RedPajama and RedPajama ahead of C4.
https://arxiv.org/html/2406.17557 recorded 2026-09-29
FineWeb paper HTML lists "C4 (172B tokens)" among the datasets compared and reports that "FineWeb shows strong performance and FineWeb-Edu (detailed below) outperforms all other open datasets we compared on our aggregate group of tasks".

Verified 2026-09-27