Back to Gap Map Organization

HAE-RAE

lab

Scores

1 product on the map — 1 closed.

KMMLU

Openness

2 high confidence
2.0
license
cc-by-nd-4.0 (card metadata and API tag)
access
public (ungated on the Hub)
dataset_card
present (card describes the benchmark, its splits and statistics, and links the paper)

The questions download freely, but the license forbids distributing changed versions, so a translation or a derived benchmark cannot be shared. The card and paper describe how the questions were collected.

Adoption

3 high confidence
3.0

Hugging Face downloads of the Hub repository over the trailing month. A download count does not say who downloaded, and evaluation harnesses can fetch a benchmark repeatedly, so the figure is a rough measure of use, not a count of users or organizations.

Capability

5 medium confidence
5.0

In its one target language, KMMLU's test has more than sixty-four hundred items that count toward depth. An item count says how much a set can test, not how precisely it ranks models.

  • https://arxiv.org/abs/2402.11548 recorded 2026-11-11

    Abstract: "We propose KMMLU, a new Korean benchmark with 35,030 expert-level multiple-choice questions across 45 subjects ranging from humanities to STEM"; "KMMLU is collected from original Korean exams".

  • https://arxiv.org/pdf/2402.11548 recorded 2026-11-11

    "all sourced from original Korean exams without any translated content"; "We predominantly source the questions from Korean License Tests"; "This filtering reduces the dataset by 34% to 243,777 questions".

  • https://datasets-server.huggingface.co/size?dataset=HAERAE-HUB/KMMLU recorded 2026-11-11

    The Hub's row counts over 45 subject configurations, complete ("partial":false): test 35,030, dev 225 and train 208,522 rows, 243,777 in all.