RedPajama-Data-v2
Together AILarge open web dataset built from Common Crawl by Together Computer, holding more than 100 billion text documents, of which around 30 billion carry quality-signal annotations and about 20 billion are deduplicated. It is published raw and unfiltered so downstream users apply their own filtering.
Openness
5 high confidence- license
- CommonCrawl-ToU+Apache-2.0(code)
- ungated
- yes
- dataset_card
- yes
Ungated with a full dataset card; redistributable per Common Crawl ToU with Apache-2.0 processing code.
- https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2 recorded 2026-09-29
HF dataset page, public with no access-request gate (its embedded page data reports gated false), shows the card text "RedPajama-V2 is an open dataset for training large language models", "The code used to load and process the dataset is licensed under the Apache 2.0 license." and a pointer to the "Common Crawl Foundation Terms of Use" for the data.
- https://huggingface.co/datasets/togethercomputer/RedPajama-Data-V2/raw/main/README.md recorded 2026-09-29
Raw dataset card says "RedPajama-V2 is an open dataset for training large language models" and its License section says "The code used to load and process the dataset is licensed under the Apache 2.0 license." and points to the "Common Crawl Foundation Terms of Use" for the data.
Adoption
3 high confidenceAdoption is measured by Hugging Face downloads of togethercomputer/RedPajama-Data-V2, the only usage channel on record for RedPajama-Data-v2.
- https://huggingface.co/api/datasets/togethercomputer/RedPajama-Data-V2 recorded 2026-10-07
11,424 downloads in the trailing 30 days for togethercomputer/RedPajama-Data-V2
Capability
3 high confidenceRedPajama-Data-v2 performs competitively in its own filtered ablation, second only to RefinedWeb on aggregate scores, but FineWeb outperforms it in FineWeb's matched comparison. It ships as raw, unfiltered web text with quality signals rather than as a pre-filtered corpus.
- https://arxiv.org/abs/2406.17557 recorded 2026-09-29
FineWeb abstract says "a 15-trillion token dataset derived from 96 Common Crawl snapshots that produces better-performing LLMs than other open pretraining datasets".
- https://arxiv.org/html/2406.17557 recorded 2026-09-29
FineWeb paper HTML lists "the deduplicated variant of RedPajama2" among the datasets compared in Fig. 10 and states that "Base FineWeb shows strong performance" against them, with "Data ablation models all had 1.71B parameters" on matched setups.
- https://arxiv.org/html/2411.12372 recorded 2026-09-29
RedPajama paper HTML reports that "fuzzy deduplication and filtering with Gopher has the highest aggregated scores across all RPv2 datasets" in its filtering ablation.
Verified 2026-09-29, except some scores, still awaiting re-confirmation