Raia Center
labScores
1 product on the map — 1 closed.
Openness
2 high confidence- license
- cc-by-nd-4.0 (card metadata; the gating terms add non-commercial, academic-research-only use)
- access
- manual (manual approval after a form sharing contact information and accepting the terms)
- dataset_card
- no (README.md exists but is empty)
Access needs a form and manual approval, and the terms allow only non-commercial academic research and forbid derivative works, including new benchmarks built from the questions. The repository has no card, so its contents are described only in the paper. The no-derivatives term alone would keep it from being open data, since a model trained on the questions is arguably a derivative.
- https://huggingface.co/api/datasets/raia-center/khayyam-challenge recorded 2026-09-24
API JSON: "gated": "manual"; tag license:cc-by-nd-4.0.
- https://huggingface.co/datasets/raia-center/khayyam-challenge recorded 2026-09-24
Gating terms: "distributed under a Creative Commons No Derivatives (CC ND) license, prohibiting the creation of derivative works ... designated exclusively for non-commercial, academic research"; "README.md exists but content is empty."
Adoption
1 high confidenceHugging Face downloads of the gated repository over the trailing month. Access requires approval, but the download count does not identify unique users or organizations, and repeated downloads are possible. Copies used by leaderboards are not counted.
- https://huggingface.co/api/datasets/raia-center/khayyam-challenge recorded 2026-09-24
43 downloads in the trailing 30 days for raia-center/khayyam-challenge
Capability
5 medium confidenceIn its one target language, Khayyam Challenge's deepest test has more than sixty-four hundred items that count toward depth. An item count says how much a set can test, not how precisely it ranks models.
- https://arxiv.org/abs/2404.06644 recorded 2026-11-08
Abstract: "Khayyam Challenge (also known as PersianMMLU), a meticulously curated collection comprising 20,192 four-choice questions sourced from 38 diverse tasks extracted from Persian examinations"; "its use of original, non-translated data tailored for Persian speakers".
- https://arxiv.org/html/2404.06644 recorded 2026-11-08
Settings: "we benchmarked a standardized prompt-0" "template on the entire dataset", with two other templates and chain-of-thought on subsets of 1000; the dataset is "originally constructed in Persian"; questions come from "Iran’s national university entrance" and "Kanoon Farhangi Amoozesh (Cultural Educational Center), wherein questions are not only verified and validated by experts in each subject". No train, dev or test split is described.
- https://huggingface.co/datasets/raia-center/khayyam-challenge recorded 2026-11-08
Hub page: "Space using raia-center/khayyam-challenge 1: ParsBench/leaderboard".