Back to Gap Map Model components / Training & synthetic datasets

peS2o

open / Overall score: 3.5

Ai2's corpus of roughly 40 million cleaned academic papers derived from S2ORC on Semantic Scholar. It is a standard open scientific-text component, documented in the pretraining mixes for OLMoE, Marin and NVIDIA's Nemotron 3.

Openness

5 high confidence

5.0

license: ODC-BY
gated: false
dataset_card: present

ODC-BY, ungated, full dataset card.

https://huggingface.co/api/datasets/allenai/peS2o recorded 2026-09-29
The Hub API record for allenai/peS2o carries the tag "license:odc-by" and the pretty name "peS2o (Pretraining Efficiently on S2ORC)", and its gated field is false (searched for gated true or manual: none).
https://huggingface.co/datasets/allenai/peS2o recorded 2026-09-29
The Hugging Face dataset page for allenai/peS2o renders the card titled "peS2o (Pretraining Efficiently on S2ORC)" with license odc-by.
https://huggingface.co/datasets/allenai/peS2o/raw/main/README.md recorded 2026-09-29
The raw dataset card front matter lists "odc-by" as the license, and the card describes "The peS2o dataset is a collection of ~40M creative open-access academic papers," with a Document Format section and a citation block.

Adoption

3 high confidence

3.0

Adoption is measured by Hugging Face downloads of allenai/peS2o, the only usage channel on record for peS2o.

https://huggingface.co/api/datasets/allenai/peS2o recorded 2026-08-13
11,867 downloads in the trailing 30 days for allenai/peS2o

Capability

4 high confidence

4.0

The standard open science corpus with documented multi-model reuse (OLMoE 57B tokens, Marin, Nemotron 3), but no ablation leadership, and OLMo 3 replaced it with a new academic-PDF dataset. The OLMoE mix card lists peS2o at 57.2B tokens and the Nemotron 3 Nano card names it among its pretraining sources; neither claims an ablation win for it.

https://huggingface.co/datasets/allenai/dolma3_pool/raw/main/README.md recorded 2026-09-29
The Dolma 3 pool card describes "The Dolma 3 pool is a dataset of over 9 trillion tokens" and lists "olmOCR Science PDFs" as its academic-documents source; searched the body for peS2o: none.
https://huggingface.co/datasets/allenai/OLMoE-mix-0924 recorded 2026-09-29
The OLMoE-mix-0924 dataset page says "The following data mix was used to train OLMoE-1B-7B" and its statistics table lists peS2o at "57.2 B" tokens and "38.8 M" docs.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 recorded 2026-09-29
The Nemotron 3 Nano model page names peS2o among the seed sources of its "Synthetic STEM from Qwen3-235B-A22B-Instruct-2507 and gpt-oss-120b" row, and lists it as "Legacy Download".

Verified 2026-09-27