Back to Gap Map Model components / Language-specific evaluation sets

SIB-200

David Adelani
open / Overall score: 2.3

SIB-200 is a topic classification benchmark in 205 languages and dialects, built by labeling the English FLORES-200 sentences with seven topics such as science, travel and politics and carrying the labels across to every aligned translation. Train, validation and test splits exist for every language. It was created by David Adelani and colleagues, with annotators recruited through Masakhane.

Openness

5 medium confidence
5.0
license
cc-by-sa-4.0 (Hugging Face license tag; the card text says "CC 4.0 Commercial" and the GitHub LICENSE is Apache-2.0 for the code.)
access
public
dataset_card
present (Describes composition and splits, but several sections are unfilled template text that wrongly calls the source news.)

The data downloads from Hugging Face without a gate under a share-alike license tag. The card's own wording of the license is loose and the GitHub license covers the code, so the tag is the clearest statement.

Adoption

3 high confidence
3.0

Hugging Face downloads over the trailing month for the SIB-200 repository. Rebuilding it from FLORES with the GitHub script is not counted.

Capability

2 medium confidence
2.0

In the typical language SIB-200 targets, its deepest test has one to four hundred items that count toward depth. Of its 195 target languages, 194 have a countable test. An item count says how much a set can test, not how precisely it ranks models.

  • https://arxiv.org/abs/2207.04672 recorded 2026-11-08

    "we evaluated the performance of over 40,000 different translation directions using a human-translated benchmark, Flores-200"

  • https://arxiv.org/abs/2309.07445 recorded 2026-11-08

    "We annotated the English portion of the dataset and extended the ... annotation to the remaining 203 languages covered in the corpus".

  • https://arxiv.org/pdf/2309.07445 recorded 2026-11-08

    "In both datasets, the source sentences were collected in English and translated by professional translators to several languages."; "Additionally, we added N’Ko—a West African language that was recently added to Flores-200 dataset (Doumbouya et al., 2023)"; Table 1 total row "701 99 204 1,004" for TRAIN, DEV and TEST; "we trained on each language in SIB-200 and evaluated on the same language"

  • https://datasets-server.huggingface.co/size?dataset=Davlan/sib200 recorded 2026-11-08

    the Hub's split sizes for Davlan/sib200: train 701, validation 99 and test 204 rows in each of the 205 language configs, ajp_Arab and apc_Arab among them

  • https://huggingface.co/datasets/Davlan/sib200/raw/main/README.md recorded 2026-11-08

    "SIB-200 is the largest publicly available topic classification dataset based on Flores-200 covering 205 languages and dialects"; English split 701/99/204.

Verified 2026-09-24