Back to Gap Map Model components / Language-specific evaluation sets

LaoBench

Beijing Academy of Artificial Intelligence (BAAI)
gated / Overall score: 3.1

LaoBench is a benchmark for testing language models in Lao, with more than 17,000 expert-curated items across three areas: culturally grounded knowledge, K-12 curriculum questions, and translation among Lao, Chinese and English. Experts wrote the items, and an agent-assisted pipeline verified them. It comes in an open subset and a held-out subset scored through a controlled service. BAAI publishes it.

The Hugging Face card carries only metadata; the description and scale come from the LaoBench paper.

Openness

2 medium confidence
2.0
license
apache-2.0 (card metadata; applies to the open subset)
access
public (open subset ungated; held-out subset evaluated only through a controlled service)
dataset_card
partial (README has YAML metadata only, no prose)
answers
held-out (paper: held-out portion enables black-box evaluation via a controlled service; the open 7k rows include answers)

The open subset downloads without a gate under Apache-2.0, while part of the benchmark is kept private and scored by BAAI's service. The repository card has no description, so how the data was made is documented only in the paper.

Adoption

1 high confidence
1.0

Adoption is Hugging Face downloads of the open subset. Evaluations on the held-out portion go through BAAI's service and are not counted.

Capability

4 medium confidence
4.0

In its one target language, LaoBench's deepest test has sixteen hundred to sixty-four hundred items that count toward depth. An item count says how much a set can test, not how precisely it ranks models.

  • https://arxiv.org/abs/2511.11334 recorded 2026-11-08

    "LaoBench contains 17,000+ expert-curated samples across three dimensions"

  • https://arxiv.org/pdf/2511.11334 recorded 2026-11-08

    "Lao-7k contains 7,000 expert-written multiple-choice questions released publicly to enable reproducible benchmarking"; "First, we perform closed-form multiple-choice evaluation on Lao-7k"; "expert linguists and domain specialists create" the questions, "writing question stems in Lao"; "All items undergo multi-stage validation combining expert review and agent-assisted verification"; Figure 3, "Distribution of Lao-7k samples across the three main categories", and "Table 4: Detailed distribution of LaoBench instances across categories and subdomains", whose counts total 7,000: "Knowledge Application" 425, 695, 566 and 314, "K12 Education" 305, 213, 764, 825 and 893, and Translation 276, 483, 388 and 853; translation instances "cover translation directions among Lao, Chinese, and English" with no count per direction; Lao-10k is "reserved for black-box evaluation"

  • https://datasets-server.huggingface.co/size?dataset=BAAI/LaoBench recorded 2026-11-08

    the Hub's split sizes: config "default", split "test", 7000 rows, the only split

  • https://huggingface.co/datasets/BAAI/LaoBench recorded 2026-11-08

    Split (1) test · 7k rows

Verified 2026-09-24