DCLM-baseline
DataComp-LM (ML Foundations)Curated pretraining dataset derived from Common Crawl, about 4 trillion tokens across 3 billion documents and roughly 7.2TB, produced by the DataComp-LM project. It was built with heuristic cleaning, Bloom-filter deduplication and fastText model-based quality filtering, and is the reference dataset for the DCLM benchmark.
Openness
5 high confidence- license
- CC-BY-4.0
- gated
- false
- dataset_card
- present
- github
- mlfoundations/dclm
Permissive CC-BY-4.0 license, ungated download, with open code repo and dataset card.
- https://huggingface.co/api/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-09-29
The Hub API record for mlfoundations/dclm-baseline-1.0 carries the tag "license:cc-by-4.0" and a gated field of false, so the download is open without an access request; the record id is "mlfoundations/dclm-baseline-1.0".
- https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-09-29
The rendered dataset page shows the card, including "DCLM-baseline is a 4T token / 3B document pretraining dataset" and the license "CC-by-4.0".
- https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0/raw/main/README.md recorded 2026-09-29
The raw dataset card lists "license: cc-by-4.0" in its metadata, a section "Dataset Details" with "Curated by:" and the description "DCLM-baseline is a 4T token / 3B document pretraining dataset".
Adoption
4 high confidenceAdoption is measured by Hugging Face downloads of mlfoundations/dclm-baseline-1.0, the only usage channel on record for DCLM-baseline.
- https://huggingface.co/api/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-08-13
309,488 downloads in the trailing 30 days for mlfoundations/dclm-baseline-1.0
Capability
5 high confidenceLeads the DataComp-LM leaderboard (beats C4/RefinedWeb/RedPajama/Dolma); current 2024-26 default of OLMoE, Marin, SmolLM3, and RWKV. DCLM-Baseline 7B reaches 64% 5-shot MMLU at 2.6T tokens, a 6.6-point MMLU gain over MAP-Neo, the prior open-data state of the art.
- https://arxiv.org/abs/2406.11794 recorded 2026-08-13
DCLM-Baseline 7B reaches 64% MMLU at 2.6T tokens; model-based filtering adds >6 MMLU over the best prior open corpus
- https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-08-13
OLMoE, SmolLM3, Marin, and RWKV built on DCLM
Verified 2026-09-27, except some scores, still awaiting re-confirmation