AfroBench
McGill NLPAfroBench is an evaluation suite for large language models covering 64 African languages, 15 tasks and 22 datasets. It carries AfriMMLU, AfriXNLI, AfriMGSM, AfriSenti, MasakhaNER, MasakhaNEWS, MasakhaPOS, AfriQA, MAFAND-MT, SALT, Uhura, InjongoIntent, NaijaRC, NollySenti, NTREX African and AfriADR, with lm-evaluation-harness configs and a public leaderboard. It is published from the McGill-NLP GitHub organization, with its data in a Masakhane collection.
Belebele, FLORES, SIB-200, MMMLU and XL-Sum are in the suite but are separate entries, so their repositories are not listed or counted here. NaijaRC and NollySenti sit in a personal Hugging Face namespace and are also left out.
Openness
4 medium confidence- license
- mixed-per-subset(afrimmlu: apache-2.0
- afrixnli
- apache-2.0
- afrimgsm
- apache-2.0
- afrisenti
- cc-by-nc-sa-2.0
- masakhaner-x
- unknown
- masakhanews
- afl-3.0
- masakhapos
- afl-3.0
- afriqa-gold-passages
- cc-by-sa-4.0
- mafand
- cc-by-nc-4.0
- uhura-arc-easy
- mit
- InjongoIntent
- apache-2.0
- ntrex_african
- cc-by-sa-4.0
- AfriADR
- apache-2.0
- salt
- cc-by-sa-4.0
- access
- public(Masakhane member repositories ungated
- dataset_card
- present(README, project site and paper describe tasks and datasets
Each member dataset keeps its own license, from Apache-2.0 and MIT to non-commercial terms on MAFAND-MT and AfriSenti, and MasakhaNER-X lists its license as unknown. Everything but SALT downloads without a gate.
- https://huggingface.co/api/collections/masakhane/afrobench-67dbf553ebf5701c2207f883 recorded 2026-09-24
Collection items include masakhane/afrimmlu, afrixnli, afrimgsm, afrisenti, masakhaner-x, masakhanews, masakhapos, afriqa-gold-passages, mafand, uhura-arc-easy, InjongoIntent, ntrex_african, AfriADR and Sunbird/salt.
- https://huggingface.co/api/datasets/masakhane/afrisenti recorded 2026-09-24
cardData license "cc-by-nc-sa-2.0".
- https://huggingface.co/api/datasets/masakhane/mafand recorded 2026-09-24
cardData license ["cc-by-nc-4.0"]; "gated": false.
- https://huggingface.co/api/datasets/masakhane/masakhaner-x recorded 2026-09-24
cardData license ["unknown"].
- https://huggingface.co/api/datasets/Sunbird/salt recorded 2026-09-24
"gated": "auto"; cardData license "cc-by-sa-4.0".
- https://raw.githubusercontent.com/McGill-NLP/AfroBench/main/README.md recorded 2026-09-24
"All scripts, prompts and datasets are opensource and available"; run via LM-Harness or API; no license section.
Adoption
1 low confidenceGitHub stars on the AfroBench repository. The member datasets it evaluates are also downloaded on their own, but those downloads are not runs of the suite, so they are left out; a star is attention rather than use.
- https://ungh.cc/repos/McGill-NLP/AfroBench recorded 2026-09-24
GitHub repository record for McGill-NLP/AfroBench (via the ungh.cc mirror of the GitHub API): 24 stargazers, last push 2025-07-28.
Capability
5 medium confidenceAfroBench brings 22 datasets across 64 African languages into one evaluation with harness tasks and a public leaderboard, and models such as InkubaLM report on its members. No other open suite here covers as many African languages and tasks. Its breadth comes from bundling existing Masakhane and partner benchmarks.
- https://arxiv.org/abs/2408.17024 recorded 2026-09-24
InkubaLM abstract reports performance on "AfriMMLU, and the AfriXnli task".
- https://mcgill-nlp.github.io/AfroBench/ recorded 2026-09-24
"Evaluating 64 African languages across 15 NLP tasks and 22 Datasets"; leaderboard and lm-eval-harness scripts.
- https://raw.githubusercontent.com/McGill-NLP/AfroBench/main/README.md recorded 2026-09-24
Run with "LM-Harness (lm_eval/tasks/afrobench)"; results submitted to "the official leaderboard".
Verified 2026-09-24