HumanEval

license

MIT(OSI-style permissive, fully redistributable)

redistributability

full(downloadable parquet, no gating)

datasheet

yes(dataset card)

size

164 problems/test split

contamination

hand-written to avoid GitHub-dump overlap (per design)

Fully open: MIT license, ungated, downloadable, with a dataset card. Textbook open-data case for this category.

• https://huggingface.co/datasets/openai/openai_humaneval recorded 2026-08-13

  `license:mit` in the repo tags; the embedded repo state reads `"gated":false`; the dataset card renders with the 164-problem test split; 285,003 downloads in the trailing 30 days

279,859 Hugging Face downloads in the trailing 30 days for openai/openai_humaneval, which puts it in the 100K-1M band, level 4 on the dataset adoption scale. That scale tops out at >1M, an order of magnitude below the ones used for software and models, because no dataset in this corpus has ever passed 10M downloads. HumanEval is the foundational code-generation benchmark: the pass@k metric originated here, and it is still reported in most code-model releases.

• https://huggingface.co/api/datasets/openai/openai_humaneval recorded 2026-08-12

  279,859 downloads in the trailing 30 days for openai/openai_humaneval

A dataset is not 'capable', so this axis is left unscored. It is worth noting that HumanEval is now largely saturated and contaminated as a discriminator, but that is a quality caveat rather than a capability score.

• https://huggingface.co/datasets/openai/openai_humaneval recorded 2026-08-13

  a static 164-problem corpus with prompt, canonical_solution and test columns; no performance, throughput or feature claim anywhere on the page, so there is nothing for the capability axis to read