DCLM pipeline

license

MIT(OSI)

source

public

core features withheld

no

Fully OSI-licensed (MIT), full source public.

• https://github.com/mlfoundations/dclm/blob/main/LICENSE recorded 2026-08-13

  MIT License text

• https://raw.githubusercontent.com/mlfoundations/dclm/main/LICENSE recorded 2026-09-29

  LICENSE file begins "MIT License" and states "Permission is hereby granted, free of charge" to any person obtaining a copy of the software.

• https://raw.githubusercontent.com/mlfoundations/dclm/main/README.md recorded 2026-09-29

  README states "licensed under the MIT License." and installs from the repo with "pip install -r requirements.txt"; searched the body for enterprise, license key, non-commercial and noncommercial: none.

This project has no PyPI package, so GitHub stars are the only usage signal published for it, and a star is not a use. It is the reference DataComp-LM harness that other curation pipelines are benchmarked against, so its importance runs well ahead of what that count can show.

• https://api.github.com/repos/mlfoundations/dclm recorded 2026-08-13

  stargazers_count = 1463 for mlfoundations/dclm

DCLM-baseline, the reference curation output of the DataComp-LM benchmark, set the open-data state of the art on MMLU when published.

• https://arxiv.org/abs/2406.11794 recorded 2026-09-29

  DataComp-LM abstract: "DCLM-Baseline enables training a 7B parameter language model from scratch to 64% 5-shot accuracy on MMLU with 2.6T training tokens" and "6.6 percentage point improvement on MMLU" over MAP-Neo, evaluated on "53 downstream evaluations".