Infrastructure / Core ML frameworks & libraries

Datasets

Datasets is a lightweight Hugging Face library providing one-line dataloaders for thousands of public datasets and efficient data preprocessing for ML workflows. It supports CSV, JSON, Parquet, audio, image, and video formats with streaming, memory-mapping, and multi-framework integration (PyTorch, TensorFlow, JAX). It is widely used to feed training and evaluation pipelines.

Verified live 2026-06-22 via primary sources. LICENSE file is the standard, unmodified Apache 2.0 text.

Openness

5 high confidence

5.0

license: Apache-2.0(OSI)
source: public
core-gated: ungated

LICENSE file is the standard, unmodified Apache 2.0 text.

https://github.com/huggingface/datasets/blob/main/LICENSE recorded 2026-06-22
Standard Apache License 2.0 full text

Adoption

5 high confidence

5.0

PyPI last_month downloads of 120,316,114 exceed the 50M+ Level 5 threshold.

https://pypistats.org/api/packages/datasets/recent recorded 2026-06-22
last_month downloads = 120,316,114

Capability

4 high confidence

4.0

Broad and central to data pipelines but scoped to data handling rather than modeling.

https://github.com/huggingface/datasets recorded 2026-06-22
Describes one-line dataloaders and efficient preprocessing

Unchanged since 2026-07-30 (last edited, not re-checked)