Back to Gap Map Model components / Language-specific datasets

Bloom Library datasets

SIL Global
gated / Overall score: 1.0

The Bloom Library datasets turn the community-written storybooks on SIL's Bloom Library into four multilingual resources: text for language modeling, image captioning, visual storytelling, and aligned speech. They cover more than 350 languages across over 30 families, many with only a few stories each. SIL Global builds Bloom, the book-authoring software behind the library.

Openness

3 high confidence
3.0
license
mixed-per-subset (Each entry carries its own Creative Commons license: CC-BY, CC-BY-SA, and several NC and ND variants.)
access
auto (Automatic Hugging Face gate; accepting the terms shares the user's email and username with the authors.)
dataset_card
present

Each story keeps the Creative Commons license its author chose, and most of those forbid commercial use or derivatives. The gate approves automatically but shares the user's contact details and adds a pledge against discriminatory use.

Adoption

1 high confidence
1.0

Hugging Face downloads over the trailing month summed across the four Bloom Library repositories. Reading the books on bloomlibrary.org is not counted.

Capability

1 medium confidence
1.0

The datasets gather community-written storybooks in hundreds of languages into text, captioning, storytelling and speech sets documented in an EMNLP paper, and their text went into the BLOOM model's training data as a small part of its corpus. With a couple of stories for most languages they are tiny beside English pretraining corpora of trillions of tokens, and a pooled corpus such as the one behind the Glot language model gathers far more text per language.

  • https://arxiv.org/abs/2210.14712 recorded 2026-11-12

    "the initial release of the Bloom Library datasets covers 363 languages across 32 language families".

  • https://arxiv.org/pdf/2210.14712 recorded 2026-11-12

    "Our bloom-speech dataset comprises 428 hours of total audio in 56 languages"; "a mean of 159 minutes and median of 31 minutes of audio per language family"; Table 1 total row: 25680 audio files, 2873 audio minutes.

  • https://huggingface.co/api/datasets/sil-ai/bloom-lm/tree/main recorded 2026-11-12

    File sizes in bytes: bloom_lm_train.json 35046861, bloom_lm_val.json 4276285, bloom_lm_test.json 4039385.

  • https://huggingface.co/datasets/sil-ai/bloom-lm recorded 2026-11-12

    "It includes data from 364 languages across 31 language families. There is a mean of 32 stories and median of 2 stories per language." "Although this data was used in the training of the"; "this dataset only represents a small portion of the data used to train that model".

Verified 2026-09-24