bigcode-dataset
BigCode ProjectPreprocessing, quality filtering, decontamination, and PII detection/anonymization for large code corpora. The filtering and PII pipeline behind The Stack and StarCoder training data.
BigCode bigcode-dataset, maintenance-quiet since August 2024. Built the filtering and PII pipeline behind The Stack and StarCoder. Verified 2026-08-13 via the GitHub README and the LICENSE body.
Openness
5 high confidence- license
- Apache-2.0(OSI)
- source
- public(bigcode-project/bigcode-dataset)
- governance
- BigCode
- core-gated
- ungated
Fully OSI-licensed (Apache-2.0), full source public.
- https://github.com/bigcode-project/bigcode-dataset/blob/main/LICENSE recorded 2026-08-13
Apache License Version 2.0 text
- https://raw.githubusercontent.com/bigcode-project/bigcode-dataset/main/README.md recorded 2026-08-13
README documents the whole pipeline building and installing from the published repo; no paid tier, enterprise edition or license key appears anywhere in it, so nothing is withheld from the source
Adoption
1 medium confidence497 GitHub stars on bigcode-project/bigcode-dataset, which bands at <1K stars, level 1 on the stars fallback scale. That scale caps at level 3, because a star is not a use. The repo has been dormant since 2024. As the pipeline behind The Stack and StarCoder it matters more than its star count suggests, but importance is not adoption, and no download, install or customer figure is published for this product, so stars remain the only honest signal and the level is directional.
- https://api.github.com/repos/bigcode-project/bigcode-dataset recorded 2026-08-12
stargazers_count = 497 for bigcode-project/bigcode-dataset
Capability
4 medium confidenceSolid code-corpus curation coverage; specialized to code, not actively developed.
- https://raw.githubusercontent.com/bigcode-project/bigcode-dataset/main/README.md recorded 2026-08-13
README lists language selection, PII detection and anonymization, decontamination and the line-length/alphanumeric quality filters used for The Stack and StarCoder
Verified 2026-08-12