KazMMLU
Mohamed bin Zayed University of Artificial IntelligenceKazMMLU tests language models on 23,000 multiple-choice questions in Kazakh and Russian, 10,969 and 12,031 respectively, reflecting Kazakhstan's bilingual education system. The questions cover STEM, humanities, and social sciences in high-school and university material, sourced from authentic educational materials and manually validated by native speakers and educators. MBZUAI researchers built it.
The Hub card is a stub from before the repository was renamed and does not describe the release; the description follows the paper.
Openness
2 high confidence- license
- cc-by-nc-4.0 (card metadata and API tag)
- access
- public (ungated on the Hub)
- dataset_card
- partial (card lists splits and columns but is titled with another repository name and truncates its subset list)
The questions download freely under a noncommercial license, which rules out commercial evaluation use. The card covers only the file layout, so how the questions were sourced is documented in the paper alone.
- https://huggingface.co/api/datasets/MBZUAI/KazMMLU recorded 2026-09-24
API JSON: "gated": false; tag license:cc-by-nc-4.0.
- https://huggingface.co/datasets/MBZUAI/KazMMLU/raw/main/README.md recorded 2026-09-24
Card metadata "license: cc-by-nc-4.0"; body titled "# MukhammedTogmanov/jana"; subset list ends "... (list other subsets)".
Adoption
1 high confidenceHugging Face downloads of the Hub repository. A download count does not say who downloaded, and evaluation harnesses can fetch a benchmark repeatedly, so the figure is a rough measure of use, not a count of users or organizations.
- https://huggingface.co/api/datasets/MBZUAI/KazMMLU recorded 2026-09-24
392 downloads in the trailing 30 days for MBZUAI/KazMMLU
Capability
5 medium confidenceIn the typical language KazMMLU targets, its deepest test has more than sixty-four hundred items that count toward depth. An item count says how much a set can test, not how precisely it ranks models.
- https://arxiv.org/abs/2502.12829 recorded 2026-11-08
Abstract: "KazMMLU, the first MMLU-style dataset specifically designed for Kazakh language. KazMMLU comprises 23,000 questions ... sourced from authentic educational materials and manually validated by native speakers and educators".
- https://arxiv.org/pdf/2502.12829 recorded 2026-11-08
"The dataset includes 10,969 Kazakh questions and 12,031 Russian questions"; "KazMMLU is sourced from authentic educational materials, including national exams, textbooks, and professional certification repositories"; "the question and answer choices remain in their original language (Kazakh or Russian)"; "every question included in KazMMLU undergoes manual validation"
- https://datasets-server.huggingface.co/size?dataset=MBZUAI/KazMMLU recorded 2026-11-08
the Hub's row counts per subject configuration, each named for its language, such as "Biology (High School in kaz)" and "Biology (High School in rus)": test 9,870 rows across the 12 Kazakh configurations and 13,019 across the 25 Russian ones; dev 3 rows in each configuration (36 Kazakh, 75 Russian); 23,000 rows in all
- https://huggingface.co/datasets/MBZUAI/KazMMLU recorded 2026-11-08
Hub page: "Number of rows: 23,000"; no models or Spaces listed as using it.
Verified 2026-09-24