SeaExam
Alibaba DAMO AcademySeaExam is a multiple-choice benchmark from DAMO-NLP-SG, the group behind SeaLLMs, for testing language models in Indonesian, Thai, Vietnamese, Chinese and English. It combines real human exam questions adapted from M3Exam, standardized to four shuffled options, with 2,850 MMLU questions machine-translated using Google Translate. Its companion SeaBench adds open-ended multi-turn and instruction-following questions in Indonesian, Thai and Vietnamese, and both feed the SeaExam leaderboard.
Openness
5 high confidence- license
- apache-2.0 (card metadata for both SeaExam and SeaBench)
- access
- public (both repos ungated)
- dataset_card
- present (describes the M3Exam adjustments and the MMLU translation; SeaBench card is brief)
Both sets download without a gate under Apache-2.0, answers included, and the SeaExam card explains how each part was built. The SeaBench card says little beyond its languages and purpose.
- https://huggingface.co/api/datasets/SeaLLMs/SeaExam recorded 2026-09-24
gated: false; license:apache-2.0
- https://huggingface.co/datasets/SeaLLMs/SeaBench/raw/main/README.md recorded 2026-09-24
license: apache-2.0; "SeaBench evaluates models' multi-turn and instruction-following abilities across Indonesian, Thai, and Vietnamese"
- https://huggingface.co/datasets/SeaLLMs/SeaExam/raw/main/README.md recorded 2026-09-24
license: apache-2.0; "All answers have been mapped to a numerical value"; "translated ... using Google Translate"
Adoption
1 high confidenceAdoption is Hugging Face downloads summed over SeaExam and SeaBench, most of them for SeaExam. Evaluations run through the project's own leaderboard code or local copies are not separately counted.
- https://huggingface.co/api/datasets/SeaLLMs/SeaBench recorded 2026-09-24
87 downloads in the trailing 30 days for SeaLLMs/SeaBench
- https://huggingface.co/api/datasets/SeaLLMs/SeaExam recorded 2026-09-24
447 downloads in the trailing 30 days for SeaLLMs/SeaExam
Capability
3 medium confidenceIn the typical language SeaExam targets, its deepest test has four hundred to sixteen hundred items that count toward depth. In one of its 4 languages a test taken from another dataset counts for less, because nothing independent backs it there. An item count says how much a set can test, not how precisely it ranks models.
- https://aclanthology.org/2025.findings-naacl.341.pdf recorded 2026-11-08
"we decide to manually collect exam questions from the SEA region (Indonesian (id), Thai (th), and Vietnamese (vi))"; "we collaborate with native linguists from the SEA region to systematically collect official region-specific exam questions"; "The final SeaExam comprises a total of 5,451 test samples", Table 1 totals 1,580 (id), 2,126 (th) and 1,745 (vi); on MMLU, "Then we translate the questions and the choices from English into Indonesian, Thai, and Vietnamese using Google Translate API" and "We call the curated benchmark MMLU-SEA"
- https://aclanthology.org/2025.findings-naacl.341/ recorded 2026-11-08
SeaExam and SeaBench: Benchmarking LLMs with Local Multilingual Questions in Southeast Asia, Liu et al.
- https://arxiv.org/pdf/2306.05179 recorded 2026-11-08
M3Exam: "we select 9 languages including English (from the US), Chinese (from China)" and the rest, ending "and Javanese (from Indonesia)"; "We then engage native speakers from each of the selected countries to collect official exam papers along with their answers"; "collecting real-world natural data of different languages instead of translating from English data"; "separate these as held-out development data"; "The remaining questions are used as test data during the experiment"
- https://datasets-server.huggingface.co/size?dataset=SeaLLMs/SeaExam recorded 2026-11-08
the Hub's split sizes per config, test: "m3exam-english" 1908, "m3exam-chinese" 644, "m3exam-indonesian" 1580, "m3exam-thai" 2126, "m3exam-vietnamese" 1745, and 2850 in each of "mmlu-english", "mmlu-chinese", "mmlu-indonesian", "mmlu-thai" and "mmlu-vietnamese"; dev: 32, 25, 27, 26 and 25 in the same m3exam configs, and 285 in each mmlu config
- https://huggingface.co/datasets/SeaLLMs/SeaExam recorded 2026-11-08
Number of rows: 23,813; Space using SeaLLMs/SeaExam: SeaLLMs/LLM_Leaderboard_for_SEA
- https://huggingface.co/datasets/SeaLLMs/SeaExam/raw/main/README.md recorded 2026-11-08
"The original M3Exam dataset is constructed with real human exam questions"; "We randomly selected 50 questions from each subject, totaling 2850 questions"; under "Translated MMLU": "We translate the original English questions to different languages to measure the cross-lingual alignment" and "These questions have been translated from English into Chinese, Indonesian, Thai, and Vietnamese using Google Translate"; configs "m3exam-indonesian", "m3exam-thai", "m3exam-vietnamese", "m3exam-english", "m3exam-chinese" and "mmlu-english", "mmlu-chinese", "mmlu-thai", "mmlu-vietnamese", "mmlu-indonesian", each with a dev and a test split
- https://raw.githubusercontent.com/DAMO-NLP-SG/SeaExam/main/README.md recorded 2026-11-08
"The leaderboard showcases results from two complementary benchmarks: SeaExam and SeaBench"; cites Findings of NAACL 2025 paper
Verified 2026-09-24