Back to Gap Map Organization

McGill NLP

lab · Canada

Scores

1 product on the map — 1 open.

AfroBench

Openness

4 medium confidence
4.0
license
mixed-per-subset (afrimmlu: apache-2.0; afrixnli: apache-2.0; afrimgsm: apache-2.0; afrisenti: cc-by-nc-sa-2.0; masakhaner-x: unknown; masakhanews: afl-3.0; masakhapos: afl-3.0; afriqa-gold-passages: cc-by-sa-4.0; mafand: cc-by-nc-4.0; uhura-arc-easy: mit; InjongoIntent: apache-2.0; ntrex_african: cc-by-sa-4.0; AfriADR: apache-2.0; salt: cc-by-sa-4.0; the GitHub repository has no LICENSE file)
access
public (Masakhane member repositories ungated; Sunbird/salt behind an automatic gate)
dataset_card
present (README, project site and paper describe tasks and datasets; each member has its own card)

Each member dataset keeps its own license, from Apache-2.0 and MIT to non-commercial terms on MAFAND-MT and AfriSenti, and MasakhaNER-X lists its license as unknown. Everything but SALT downloads without a gate.

Adoption

1 low confidence
1.0

GitHub stars on the AfroBench repository. The member datasets it evaluates are also downloaded on their own, but those downloads are not runs of the suite, so they are left out; a star is attention rather than use.

Capability

not assessed

In the typical language AfroBench targets, its deepest test has four hundred to sixteen hundred items, but in 29 of its 64 languages that test comes from another dataset with nothing independent in the language to back it, and 5 languages have no countable test at all. Whether the typical language has fewer than a hundred corroborated items or four hundred or more depends on where those 5 would land, so the map shows no capability score for now and its overall score reflects adoption alone.