MATH
Hendrycks et al. (UC Berkeley)Benchmark of 12,500 competition mathematics problems, split into 7,500 training and 5,000 test problems, each with a full step-by-step solution. It accompanies the paper Measuring Mathematical Problem Solving With the MATH Dataset.
Openness
1 medium confidence- license
- MIT(declared)
- access
- closed(canonical HF repo access-disabled under a DMCA takedown)
- data
- NOT downloadable from canonical HF repo (access disabled, DMCA takedown)
- datasheet
- present(card still documents schema)
- redistributability
- BLOCKED on the primary registry
MIT is the declared license, but the canonical Hugging Face repository is disabled under a DMCA takedown over AMC/AIME problem rights, so the corpus is no longer distributed from its primary registry. Data that is not distributed counts as closed whatever its license, although mirrors and eval harnesses still embed the problems.
- https://huggingface.co/datasets/hendrycks/competition_math recorded 2026-09-29
The page reads "Access to this dataset has been disabled" with "DMCA Takedown notice - see" a discussion link, while the metadata still carries "license:mit" and the card title "Mathematics Aptitude Test of Heuristics (MATH)".
Adoption
4 medium confidenceMATH is measured by Hugging Face downloads of the EleutherAI/hendrycks_math mirror, the distribution the product declares. The canonical hendrycks/competition_math repository is disabled and reports no downloads in the trailing 30 days, so it is not counted. The paper is cited thousands of times on Semantic Scholar, and OLMo 3's model card lists MATH in its evaluation table beside AIME 2024/2025, GPQA, MMLU, IFEval and LiveCodeBench.
- https://api.semanticscholar.org/graph/v1/paper/arXiv:2103.03874?fields=title,citationCount recorded 2026-09-29
The Semantic Scholar record for the MATH paper reads "Measuring Mathematical Problem Solving With the MATH Dataset" with "citationCount": 6499.
- https://huggingface.co/allenai/Olmo-3-7B-Instruct/raw/main/README.md recorded 2026-09-29
evaluation table row "| **Math** | MATH | 65.1 | 79.6 | 87.3 | 82.3 | 91.6 | 71.0 | 30.1 | 21.9 | 67.3 |", reported beside AIME 2024, AIME 2025, OMEGA, BigBenchHard, GPQA, MMLU, IFEval, LiveCodeBench v3 and others - MATH carried as a headline metric in a frontier open model release
- https://huggingface.co/api/datasets/EleutherAI/hendrycks_math recorded 2026-09-29
The Hub API record for the declared EleutherAI mirror, whose description reads "MATH dataset from https://github.com/hendrycks/math", is not disabled and reports 183,494 downloads in the trailing 30 days (downloads field).
- https://huggingface.co/api/datasets/hendrycks/competition_math recorded 2026-09-29
The Hub API record for the canonical repo, whose description reads "Each problem in MATH has a full step-by-step solution", reports 0 downloads in the trailing 30 days (downloads field) and disabled true.
- https://huggingface.co/datasets/hendrycks/competition_math recorded 2026-09-29
The canonical dataset page reads "Access to this dataset has been disabled"; its embedded metadata reports 0 downloads in the trailing 30 days and 1,034,712 all-time.
Capability
not assessedA dataset is not 'capable', so capability is left unscored.
- https://huggingface.co/datasets/hendrycks/competition_math recorded 2026-09-29
Searched the body for accuracy, throughput and SOTA claims: none; the card describes "problems from mathematics competitions, including the AMC 10, AMC 12, AIME, and more" with "a full step-by-step solution".
Verified 2026-09-29