datamap-rs
Allen Institute for AIRust JSONL map-filter-annotate pipeline for cleaning and filtering large text corpora. The data-cleaning stage of the Dolma 3 pipeline.
Ai2 datamap-rs; repo active (pushed 2026-03), Apache-2.0 confirmed live 2026-07-21. Cleaning/filtering stage of the Dolma 3 pipeline behind OLMo 3.
Openness
5 high confidence5.0
- license
- Apache-2.0(OSI)
- source
- public(allenai/datamap-rs)
- governance
- Allen Institute for AI
- core-gated
- ungated
Fully OSI-licensed (Apache-2.0), full source public.
- https://github.com/allenai/datamap-rs/blob/main/LICENSE recorded 2026-07-21
Apache License Version 2.0 text
Adoption
1 medium confidence1.0
New (2026) single-purpose Rust CLI, ~56 GitHub stars; real but niche, used in the Dolma 3 pipeline.
- https://github.com/allenai/datamap-rs recorded 2026-07-21
56 stars
Capability
3 medium confidence3.0
One capability slot (cleaning) of the Dolma 3 toolchain.
- https://github.com/allenai/datamap-rs recorded 2026-07-21
datamap-rs repo feature set confirmed
Unchanged since 2026-07-30 (last edited, not re-checked)