FineWeb-Edu
Hugging FaceEducational subset of FineWeb holding about 1.3 trillion tokens, produced by filtering FineWeb with an educational-quality classifier trained on Llama-3-70B annotations and keeping the text scored as highly educational. It targets knowledge- and reasoning-heavy evaluation relative to unfiltered web data.
Openness
5 high confidence- license
- ODC-BY
- gated
- false
- dataset_card
- present
- format
- parquet
Permissive ODC-BY license with ungated download and full dataset card.
- https://huggingface.co/api/datasets/HuggingFaceFW/fineweb-edu recorded 2026-09-29
Hub API record lists "task_categories:text-generation" and "license:odc-by" and "format:parquet" among its tags, with gated, private and disabled all false.
- https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu recorded 2026-09-29
Dataset page renders the license sentence "Open Data Commons Attribution License (ODC-By) v1.0" on the card.
- https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu/raw/main/README.md recorded 2026-09-29
Raw card opens with front matter "license: odc-by" and describes "1.3 trillion tokens of the finest educational data". Searched the card for extra_gated and access-request text: none.
Adoption
4 high confidenceAdoption is measured by Hugging Face downloads of HuggingFaceFW/fineweb-edu, the only usage channel on record for FineWeb-Edu.
- https://huggingface.co/api/datasets/HuggingFaceFW/fineweb-edu recorded 2026-08-13
395,837 downloads in the trailing 30 days for HuggingFaceFW/fineweb-edu
Capability
5 high confidenceFineWeb-Edu raises MMLU and ARC well above unfiltered FineWeb, and matches the MMLU score of the Matrix corpus with almost 10x fewer tokens. SmolLM2 and SmolLM3 both use it as a core web component alongside DCLM.
- https://arxiv.org/abs/2406.17557 recorded 2026-09-29
arXiv abstract for the FineWeb paper states "LLMs pretrained on FineWeb-Edu exhibit dramatically better performance on knowledge- and reasoning-intensive benchmarks like MMLU and ARC".
- https://arxiv.org/html/2406.17557 recorded 2026-09-29
FineWeb paper HTML says "On MMLU, FineWeb-Edu can match the final performance of Matrix with almost 10x fewer tokens, demonstrating the effectiveness of classifiers trained on LLM annotations for large-scale data filtering" and that "ARC score goes from 46% to 57%".
- https://arxiv.org/html/2502.02737 recorded 2026-09-29
SmolLM2 paper HTML says "We find that FineWeb-Edu achieves higher scores on the educational benchmarks MMLU, ARC, and OpenBookQA, while DCLM performs better on HellaSwag and CommonsenseQA" and "we found that a 60% FineWeb-Edu and 40% DCLM mix works well".
- https://huggingface.co/blog/smollm3 recorded 2026-09-29
SmolLM3 blog lists the stage-one web mixture as "Web: 85% (12% multilingual) - FineWeb-Edu, DCLM, FineWeb2 and FineWeb2-HQ".
Verified 2026-09-27