Back to Gap Map Model components / Training & synthetic datasets

OpenWebMath

OpenWebMath project
open / Overall score: 3.0

Corpus of 14.7 billion tokens of mathematical web text extracted from Common Crawl with LaTeX preserved, released by researchers at the University of Toronto and collaborators. It appears in Proof-Pile-2 behind Llemma, and in OLMoE, Nemotron 3 and StarCoder2's math supplements.

Openness

5 high confidence
5.0
license
ODC-BY
gated
false
dataset_card
present

ODC-BY 1.0, ungated, full dataset card.

Adoption

3 high confidence
3.0

Adoption is measured by Hugging Face downloads of the dataset over the trailing month, the only usage channel on record for OpenWebMath.

Capability

3 high confidence
3.0

OpenWebMath is used in Proof-Pile-2 behind Llemma and in the OLMoE and StarCoder2 math mixes, but newer math corpora beat it in ablations. Every FineMath subset outperforms it on GSM8K, MATH and MMLU-STEM in the SmolLM2 ablations, and Nemotron-CC-Math outperforms it as well.

  • https://arxiv.org/html/2502.02737v1 recorded 2026-09-29

    The SmolLM2 paper reports "All FineMath subsets consistently outperform OWM and InfiMM-WebMath on GSM8K, MATH, and MMLU-STEM".

  • https://arxiv.org/html/2508.15096v1 recorded 2026-09-29

    The Nemotron-CC-Math paper states "Nemotron-CC-Math-4+ not only surpasses all prior open math datasets" and that variants "consistently outperform or obtain comparable results to OpenWebMath, MegaMath, and FineMath baselines".

Verified 2026-09-27