Back to Gap Map Model components / Training & synthetic datasets

TxT360

open / Overall score: 3.0

LLM360's globally deduplicated pretraining corpus, combining 99 Common Crawl snapshots with 14 curated sources including FreeLaw, PG-19, arXiv, Wikipedia and StackExchange. It holds about 5 trillion deduplicated tokens and supports a recipe reaching beyond 15 trillion, and trains K2-V2 together with its mid-training and SFT extensions.

A family record covering TxT360 with the Midas mid-training and 3efforts SFT extensions. The LLM360 path now redirects to the IFM organization.

Openness

5 high confidence

5.0

license: ODC-BY
gated: false
dataset_card: present

ODC-BY, ungated, documented pipeline on GitHub. The LLM360 path redirects to the IFM org on the Hub, same repo.

https://huggingface.co/datasets/LLM360/TxT360 recorded 2026-09-29
Hub page for LLM360/TxT360 (redirects to IFM/TxT360) whose embedded page data carries gated":false, meaning no access gate, and lists "The collection for K2-V2 models."
https://huggingface.co/datasets/LLM360/TxT360/raw/main/README.md recorded 2026-09-29
Raw dataset card with front matter "license: odc-by" and a "TxT360 Performance" section stating "we sampled 1.5T tokens from both FineWeb and TxT360" and "a training ablation on an 8x8B Mixture-of-Experts architecture" with a plot but no numeric scores in the text.

Adoption

2 high confidence

2.0

Adoption is measured by Hugging Face downloads of IFM/TxT360, the renamed LLM360 organization's repository and the only usage channel on record for TxT360.

https://huggingface.co/api/datasets/LLM360/TxT360 recorded 2026-08-13
9,230 downloads in the trailing 30 days for IFM/TxT360

Capability

4 medium confidence

4.0

A self-reported controlled ablation beats FineWeb (8x8B MoE, 1.5T tokens) but without published numeric scores; trains K2 / K2-V2 (LLM360, 2024-25). The card describes ~5T globally deduplicated tokens and a mixing recipe reaching 15T+, and the K2-V2 collection lists it.

https://huggingface.co/datasets/LLM360/TxT360 recorded 2026-09-29
Hub page for LLM360/TxT360 (redirects to IFM/TxT360) listing "The collection for K2-V2 models."
https://huggingface.co/datasets/LLM360/TxT360/raw/main/README.md recorded 2026-09-29
Raw dataset card describing "~5T tokens of high-quality data" from global deduplication, and a recipe that "will create a dataset of 15T+ tokens", with the 1.5T-token FineWeb comparison ablation "on an 8x8B Mixture-of-Experts architecture" shown only as a chart.

Verified 2026-09-29, except some scores, still awaiting re-confirmation