DCLM-baseline
DataComp-LM (ML Foundations)Curated pretraining dataset derived from Common Crawl, about 4 trillion tokens across 3 billion documents and roughly 7.2TB, produced by the DataComp-LM project. It was built with heuristic cleaning, Bloom-filter deduplication and fastText model-based quality filtering, and is the reference dataset for the DCLM benchmark.
OLMoE, SmolLM3, Marin and RWKV were built on it. Verified 2026-08-13 via the mlfoundations/dclm-baseline-1.0 dataset card on Hugging Face.
Openness
5 high confidence- license
- CC-BY-4.0
- gated
- false
- dataset_card
- present
- github
- mlfoundations/dclm
Permissive CC-BY-4.0 license, ungated download, with open code repo and dataset card.
- https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-08-13
Dataset card, CC-BY-4.0 license, ungated public download, GitHub link
Adoption
4 high confidence309,488 downloads in the trailing 30 days for mlfoundations/dclm-baseline-1.0, which bands at 100K-1M, level 4 on the dataset adoption scale.
- https://huggingface.co/api/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-08-13
309,488 downloads in the trailing 30 days for mlfoundations/dclm-baseline-1.0
Capability
5 high confidenceLeads the DataComp-LM leaderboard (beats C4/RefinedWeb/RedPajama/Dolma); current 2024-26 default of OLMoE, Marin, SmolLM3, and RWKV. DCLM-Baseline 7B reaches 64% 5-shot MMLU at 2.6T tokens, a 6.6-point MMLU gain over MAP-Neo, the prior open-data state of the art.
- https://arxiv.org/abs/2406.11794 recorded 2026-08-13
DCLM-Baseline 7B reaches 64% MMLU at 2.6T tokens; model-based filtering adds >6 MMLU over the best prior open corpus
- https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0 recorded 2026-08-13
OLMoE, SmolLM3, Marin, and RWKV built on DCLM
Verified 2026-08-13