Back to Gap Map Model components / Language-specific evaluation sets

VMLU

Zalo AI
gated / Overall score: 1.0

VMLU is a Vietnamese benchmark suite for language models from Zalo AI and JAIST. Its core multiple-choice set has 10,880 questions across 58 subjects in STEM, humanities, social sciences and professional fields, drawn from school, university and national graduation examinations, spanning elementary school to professional study. The suite also adds reading-comprehension, discrete-reasoning and dialogue sets, and scores submissions on a public leaderboard at vmlu.ai.

Openness

2 medium confidence
2.0
license
not-clearly-stated-on-card (README 'Licenses: TBU'; repo LICENSE.txt is MIT for 'the Software'; vmlu.ai names no data license)
access
public (zip downloads offered on vmlu.ai; not on Hugging Face)
dataset_card
present (README gives subject table, format and examples; site describes sources)
answers
held-out (test predictions are submitted as an id,answer CSV to vmlu.ai/submit after login; dev examples carry answers)

Questions can be downloaded from the project site, and test scoring runs through a submission page that requires login. No license for the data itself is published: the repository's MIT file covers the code, and its license section is unfinished.

Adoption

1 low confidence
1.0

GitHub stars on the VMLU repository; the benchmark is also served from vmlu.ai, which publishes no download count, and a star is attention rather than use.

Capability

not assessed

No source says how many questions are in VMLU's official test split, so its depth cannot be counted and the map shows no capability score for now. Its overall score reflects adoption alone.

Verified 2026-09-24