Back to Gap Map Model components / Training & synthetic datasets

FineWeb-2

open / Overall score: 4.0(strong)

Multilingual successor to FineWeb, covering more than a thousand languages and writing systems with over a trillion tokens of filtered and deduplicated Common Crawl text. Each language runs through its own pipeline with deduplication and quality filtering, extending the FineWeb work beyond English.

Openness

5 high confidence

5.0

license: ODC-BY
gated: false
dataset_card: present
languages: 1000+

Permissive ODC-BY license, ungated, with a dataset card spanning 1000+ languages.

https://huggingface.co/api/datasets/HuggingFaceFW/fineweb-2 recorded 2026-09-29
Hub API record for HuggingFaceFW/fineweb-2 has its gated field false and private false, and its card description reads "A sparkling update with 1000s of languages".
https://huggingface.co/datasets/HuggingFaceFW/fineweb-2 recorded 2026-09-29
Public Hub page for the dataset says "available under the permissive" ODC-By 1.0 license and "over 1000" languages.
https://huggingface.co/datasets/HuggingFaceFW/fineweb-2/raw/main/README.md recorded 2026-09-29
Raw card front matter sets "license: odc-by" and the license section says "Open Data Commons Attribution License (ODC-By) v1.0"; searched for extra_gated, gated: none.

Adoption

3 high confidence

3.0

Adoption is measured by Hugging Face downloads of the dataset over the trailing month, the only usage channel on record for FineWeb-2.

https://huggingface.co/api/datasets/HuggingFaceFW/fineweb-2 recorded 2026-08-13
82,352 downloads in the trailing 30 days for HuggingFaceFW/fineweb-2

Capability

5 high confidence

5.0

FineWeb-2 is the multilingual web source for Apertus, which was pretrained on 15T tokens covering 1811 languages. The dataset card reports that FineWeb-2 outperforms CC-100, mC4, CulturaX and HPLT on FineTasks in ablations across nine languages.

https://arxiv.org/abs/2506.20920 recorded 2026-09-29
Abstract page of the FineWeb2 paper says "We extensively ablate our pipeline design choices on a set of nine diverse languages".
https://arxiv.org/html/2509.14233 recorded 2026-09-29
Apertus paper HTML full text: "on 15T tokens from 1811 languages during pretraining, taken from the FineWeb-2 web crawl dataset".
https://huggingface.co/datasets/HuggingFaceFW/fineweb-2 recorded 2026-08-13
FineWeb-2 outperforms CulturaX/mC4/CC-100/HPLT on FineTasks in controlled ablations
https://huggingface.co/datasets/HuggingFaceFW/fineweb-2/raw/main/README.md recorded 2026-09-29
Card says FineWeb2 "outperforms other popular pretraining datasets covering multiple languages (such as CC-100, mC4, CulturaX or HPLT".

Verified 2026-09-27