Back to Gap Map Model components / Training & synthetic datasets

Cosmopedia

open / Overall score: 3.0

Synthetic pretraining dataset of more than 31 million textbooks, blog posts, stories and educational articles, about 25 billion tokens, generated with Mixtral-8x7B-Instruct from seed samples drawn from web data, Stanford courses, OpenStax, Khan Academy and WikiHow. It was built by Hugging Face's Smol Models Research team as a reference corpus for synthetic-data pretraining recipes.

Cosmopedia v2 is the direct successor.

Openness

5 high confidence

5.0

license: apache-2.0
access: ungated
dataset_card: present

Permissive Apache-2.0 license, ungated, full dataset card and downloadable parquet files.

https://huggingface.co/api/datasets/HuggingFaceTB/cosmopedia recorded 2026-09-29
The Hub API record for the dataset reports the field gated as false, lists the tag "license:apache-2.0" and serves parquet files such as "data/auto_math_text/train-00000-of-00018.parquet".
https://huggingface.co/datasets/HuggingFaceTB/cosmopedia recorded 2026-09-29
The dataset page is published by "Hugging Face Smol Models Research" and its embedded metadata carries the license value "apache-2.0".
https://huggingface.co/datasets/HuggingFaceTB/cosmopedia/raw/main/README.md recorded 2026-09-29
The raw dataset card front matter declares "license: apache-2.0" and the card describes "over **30 million files** and **25 billion tokens**" with per-split documentation.

Adoption

3 high confidence

3.0

Adoption is measured by Hugging Face downloads of the dataset over the trailing month, the only usage channel on record for Cosmopedia.

https://huggingface.co/api/datasets/HuggingFaceTB/cosmopedia recorded 2026-08-13
22,057 downloads in the trailing 30 days for HuggingFaceTB/cosmopedia

Capability

3 high confidence

3.0

Cosmo-1B beats TinyLlama on ARC and MMLU but trails Phi-1.5. Cosmopedia v2 has since superseded it.

https://huggingface.co/blog/cosmopedia recorded 2026-09-29
The Hugging Face blog post says the Cosmo-1B model "performs better than TinyLlama 1.1B on ARC-easy, ARC-challenge, OpenBookQA, and MMLU" and that "we notice some performance gaps compared to Phi-1.5".
https://huggingface.co/datasets/HuggingFaceTB/cosmopedia-v2 recorded 2026-09-29
The smollm-corpus page, where the cosmopedia-v2 URL now redirects, says "Cosmopedia v2 is an enhanced version of Cosmopedia" and lists it as a dataset subset.

Verified 2026-09-27