DarijaMMLU
Mohamed bin Zayed University of Artificial IntelligenceDarijaMMLU tests language models on 22,027 multiple-choice questions in Moroccan Darija across 44 subjects. The questions are selected subsets of MMLU and ArabicMMLU, machine-translated into Darija with Claude 3.5 Sonnet and then manually reviewed and adapted, with technical and culturally inappropriate topics left out. MBZUAI-Paris built it as part of the Atlas-Chat evaluation suite.
Openness
5 medium confidence- license
- mit(card metadata
- access
- public(ungated on the Hub)
- dataset_card
- present(card describes sources, translation model and review)
The questions download freely and the card states where each one came from and how it was translated. Whether the permissive grant can cover the ArabicMMLU-derived questions is unclear, since the source benchmark forbids commercial use.
- https://huggingface.co/api/datasets/MBZUAI-Paris/DarijaMMLU recorded 2026-09-24
API JSON: "gated": false, "private": false.
- https://huggingface.co/datasets/MBZUAI-Paris/DarijaMMLU/raw/main/README.md recorded 2026-09-24
Card metadata "license: mit"; Licensing Information: "[MIT License](https://github.com/hendrycks/test/blob/master/LICENSE)"; each row has a "source" field (MMLU or ArabicMMLU).
Adoption
2 high confidenceHugging Face downloads of the Hub repository. Evaluation harnesses fetch a benchmark on each run, so the figure counts runs, not the groups using it.
- https://huggingface.co/api/datasets/MBZUAI-Paris/DarijaMMLU recorded 2026-09-24
3008 downloads in the trailing 30 days for MBZUAI-Paris/DarijaMMLU
Capability
2 medium confidenceDarijaMMLU gives Moroccan Darija a broad knowledge benchmark and is the main test for the Atlas-Chat models. Its questions are machine translations of MMLU and ArabicMMLU with manual review, not material written in Darija, whereas KazMMLU is built from native educational material.
- https://arxiv.org/abs/2409.17912 recorded 2026-09-24
Abstract: "our 9B model gains a 13% performance boost over a larger 13B model on DarijaMMLU, in our newly introduced evaluation suite for Darija".
- https://huggingface.co/datasets/MBZUAI-Paris/DarijaMMLU/raw/main/README.md recorded 2026-09-24
"The selected subsets were translated into Darija using Claude 3.5 Sonnet." "created through a combination of machine translation and manual review".
Verified 2026-09-24