AI Potluck
Contribute
Back to Gap Map Model components / Language-specific datasets

VMLU

Zalo AI
gated / Overall score: 3.1

VMLU is a Vietnamese benchmark suite for language models from Zalo AI and JAIST. Its core multiple-choice set has 10,880 questions across 58 subjects in STEM, humanities, social sciences and professional fields, drawn from school, university and national graduation examinations, spanning elementary school to professional study. The suite also adds reading-comprehension, discrete-reasoning and dialogue sets, and scores submissions on a public leaderboard at vmlu.ai.

Openness

2 medium confidence
2.0
license
not-clearly-stated-on-card(README 'Licenses: TBU'
access
public(zip downloads offered on vmlu.ai
dataset_card
present(README gives subject table, format and examples
answers
held-out(test predictions are submitted as an id,answer CSV to vmlu.ai/submit after login

Questions can be downloaded from the project site, and test scoring runs through a submission page that requires login. No license for the data itself is published: the repository's MIT file covers the code, and its license section is unfinished.

Adoption

1 low confidence
1.0

GitHub stars on the VMLU repository; the benchmark is also served from vmlu.ai, which publishes no download count, and a star is attention rather than use.

Capability

4 medium confidence
4.0

VMLU gathers Vietnamese exam questions in 58 subjects and runs a public leaderboard that scores open and API models. It covers one language and one kind of task, where SEA-HELM spans many task types across eleven languages, and its documentation is a README and website rather than a paper.

Verified 2026-09-24