Model components / Training & synthetic datasets

peS2o

peS2o is Ai2's ODC-BY corpus of ~40M cleaned academic papers derived from S2ORC (Semantic Scholar). It is a standard open scientific-text component, documented in OLMoE, Marin, and NVIDIA's Nemotron 3 pretraining data.

Openness

5 high confidence

5.0

license: ODC-BY
gated: false
dataset_card: present

ODC-BY, ungated, full dataset card.

https://huggingface.co/datasets/allenai/peS2o recorded 2026-07-04
ODC-BY license, ungated, dataset card

Adoption

3 high confidence

3.0

11,347 monthly downloads on Hugging Face, graded on the training-corpus bands.

https://huggingface.co/api/datasets/allenai/peS2o recorded 2026-07-04
downloads field = 11,347 (30-day)

Capability

4 high confidence

4.0

The standard open science corpus with documented multi-model reuse (OLMoE 57B tokens, Marin, Nemotron 3), but no ablation leadership, and OLMo 3 replaced it with a new academic-PDF dataset.

https://huggingface.co/datasets/allenai/OLMoE-mix-0924 recorded 2026-07-04
peS2o contributes 57.2B tokens to the OLMoE mix and is used in Marin and Nemotron 3
https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 recorded 2026-07-04
Nemotron 3 Nano lists peS2o among pretraining sources

Unchanged since 2026-07-04 (last edited, not re-checked)