Aaron Gokaslan (Skylion007)
individualScores
1 product on the map — 1 open.
Openness
5 high confidence- license
- cc0-1.0 (covers the packaging, while the creators claim no rights in the extracted text)
- ungated
- yes
- dataset_card
- present
OpenWebText downloads without a sign-in, with a card describing how it was collected, and its creators release it under CC0. The text itself comes from third-party web pages, which the creators do not claim to own and will remove on request.
- https://huggingface.co/api/datasets/Skylion007/openwebtext recorded 2026-11-12
Dataset API for Skylion007/openwebtext: `gated: false`, the tag `license:cc0-1.0`, and 80 parquet shards of plain_text/train.
- https://huggingface.co/datasets/Skylion007/openwebtext/raw/main/README.md recorded 2026-11-12
Dataset card with front matter `license: cc0-1.0`, a Source Data section on how the Reddit-linked pages were collected and filtered, and the licensing note "We do not own any of the text from which these data has been extracted. We license the actual packaging of these parallel data under the Creative Commons CC0 license".
- https://skylion007.github.io/OpenWebTextCorpus/ recorded 2026-11-12
Project page: "This distribution was created by Aaron Gokaslan and Vanya Cohen of Brown University", with the same CC0 packaging license and a notice-and-take-down policy.
Adoption
3 high confidenceHugging Face downloads of the dataset over the trailing month, the only usage channel on record for OpenWebText.
- https://huggingface.co/api/datasets/Skylion007/openwebtext recorded 2026-11-12
"downloads":41908 for Skylion007/openwebtext, the trailing 30-day count the Hub API reports.
Capability
3 medium confidenceOpenWebText helped train RoBERTa and DeBERTa and remains the usual corpus for reproducing GPT-2 at small scale. Like C4, it is a 2019-era English web corpus that has proven its worth but is no longer a first choice for pretraining.
- https://arxiv.org/html/2006.03654 recorded 2026-11-12
DeBERTa paper names "OPENWEBTEXT (public Reddit content (Gokaslan & Cohen, 2019); 38GB)" in its pretraining data.
- https://huggingface.co/FacebookAI/roberta-base/raw/main/README.md recorded 2026-11-12
RoBERTa model card lists OpenWebText, "an opensource recreation of the WebText dataset used to train GPT-2", among the five datasets RoBERTa was pretrained on.
- https://raw.githubusercontent.com/karpathy/nanoGPT/master/README.md recorded 2026-11-12
nanoGPT README: "the file `train.py` reproduces GPT-2 (124M) on OpenWebText".