Back to Gap Map Model components / Benchmark / eval datasets

MATH

Hendrycks et al. (UC Berkeley)
closed / Overall score: 4.0(strong)

Benchmark of 12,500 competition mathematics problems, split into 7,500 training and 5,000 test problems, each with a full step-by-step solution. It accompanies the paper Measuring Mathematical Problem Solving With the MATH Dataset.

Openness

1 medium confidence
1.0
license
MIT(declared)
access
closed(canonical HF repo access-disabled under a DMCA takedown)
data
NOT downloadable from canonical HF repo (access disabled, DMCA takedown)
datasheet
present(card still documents schema)
redistributability
BLOCKED on the primary registry

MIT is the declared license, but the canonical Hugging Face repository is disabled under a DMCA takedown over AMC/AIME problem rights, so the corpus is no longer distributed from its primary registry. Data that is not distributed counts as closed whatever its license, although mirrors and eval harnesses still embed the problems.

  • https://huggingface.co/datasets/hendrycks/competition_math recorded 2026-09-29

    The page reads "Access to this dataset has been disabled" with "DMCA Takedown notice - see" a discussion link, while the metadata still carries "license:mit" and the card title "Mathematics Aptitude Test of Heuristics (MATH)".

Adoption

4 medium confidence
4.0

MATH is measured by Hugging Face downloads of the EleutherAI/hendrycks_math mirror, the distribution the product declares. The canonical hendrycks/competition_math repository is disabled and reports no downloads in the trailing 30 days, so it is not counted. The paper is cited thousands of times on Semantic Scholar, and OLMo 3's model card lists MATH in its evaluation table beside AIME 2024/2025, GPQA, MMLU, IFEval and LiveCodeBench.

Capability

not assessed

A dataset is not 'capable', so capability is left unscored.

Verified 2026-09-29