Back to Gap Map Model components / Training & synthetic datasets

Common Corpus

open / Overall score: 3.5

Multilingual pretraining dataset of roughly 2.27 trillion tokens, assembled by PleIAs with partners including the AI Alliance from uncopyrighted or openly licensed content only. It is organized into six curated collections - public-domain books, government documents, academic content, code, semantic data and web text - with document-level provenance and toxicity filtering. Its card reports 33 languages with more than 1 billion tokens each.

Openness

5 high confidence

5.0

license: open(uncopyrighted/freely-licensed only)
ungated: yes
dataset_card: yes

Ungated with a full dataset card and built exclusively from uncopyrighted or openly licensed sources.

https://huggingface.co/datasets/PleIAs/common_corpus recorded 2026-09-29
Hub dataset page for PleIAs/common_corpus, which renders the card with no access-request prompt and carries the line "Common Corpus is the largest open licensed text dataset, comprising 2.27 trillion tokens". Searched the body for a contact-information agreement prompt and a request-access button: none.
https://huggingface.co/datasets/PleIAs/common_corpus/raw/main/README.md recorded 2026-09-29
Raw dataset card describing Common Corpus as "Truly Open" and stating "All data in Common Corpus are either uncopyrighted or freely licensed and may be used for both commercial and non-commercial purposes", with a per-document license field ("sharing rights for the content either uncopyrighted") and a usage snippet.

Adoption

4 high confidence

4.0

Adoption is measured by Hugging Face downloads of PleIAs/common_corpus, the only usage channel on record for Common Corpus.

https://huggingface.co/api/datasets/PleIAs/common_corpus recorded 2026-10-07
178,998 downloads in the trailing 30 days for PleIAs/common_corpus

Capability

3 high confidence

3.0

Trains the Pleias 1.0 SLM family and several European models (2024-25), but small demonstrator models with no controlled ablation wins; leads on ethical sourcing rather than capability. The corpus is 2.27 trillion tokens.

https://arxiv.org/html/2506.01732v1 recorded 2026-09-29
Technical report stating "seven European SLMs and LLMs have been trained on at least some part of Common Corpus" and that the Pleias 1.0 SLMs "were exclusively trained on Common Corpus". Searched the body for ablation: none, so no controlled ablation against other corpora is reported.
https://huggingface.co/blog/Pclanglais/common-corpus recorded 2026-09-29
Release blog post headlined "Releasing Common Corpus: the largest public domain dataset for training LLMs".

Verified 2026-09-29, except some scores, still awaiting re-confirmation