TxT360 (pipeline)
LLM360Construction pipeline for the TxT360 open pretraining corpus: global deduplication and source processing across 99 Common Crawl snapshots and 14 curated sources. Distinct from the released TxT360 dataset.
Openness
2 low confidence- license
- none-declared(the README badge asserts Apache-2.0 but the repo ships no LICENSE file and the GitHub license endpoint 404s)
- source
- public(LLM360/TxT360)
The README badge claims Apache-2.0, but the repository ships no LICENSE file: the GitHub license endpoint returns 404, the repository API reports no license, and the badge links to a LICENSE in a different repository. That leaves no actual redistribution grant, so the code can be read but not freely reused, which the record treats as no license declared rather than as the badge's claim. No license is declared anywhere the publisher could declare one, so the license tier is unstated.
- https://api.github.com/repos/LLM360/TxT360 recorded 2026-08-13
the repository API reports a null license for LLM360/TxT360, so the repo still ships no LICENSE file; public, not archived, last push 2024-12-18
- https://github.com/LLM360/TxT360 recorded 2026-08-13
README asserts Apache-2.0; no LICENSE file in repo (GitHub reports null license)
Adoption
1 medium confidenceGitHub stars are the only usage signal published for this repository, and a star is not a use. What matters is the 5.7-trillion-token corpus these scripts build, not the tool's own adoption.
- https://api.github.com/repos/LLM360/TxT360 recorded 2026-08-13
stargazers_count = 25 for LLM360/TxT360
Capability
3 medium confidenceReplication-oriented construction scripts rather than an evolving tool.
- https://raw.githubusercontent.com/LLM360/TxT360/main/README.md recorded 2026-09-29
README states "This repository contains the scripts for replicating TxT360" and structures the scripts as "common-crawl", "curated-sources" and "deduplication", with "Run the deduplication scripts to globally deduplicate data from both Common Crawl and curated sources."
Verified 2026-09-27, except some scores, still awaiting re-confirmation