AI Potluck
Model components / Benchmark / eval datasets

APPS

Hendrycks et al. (UC Berkeley)

Automated Programming Progress Standard: 10,000 coding problems split evenly between train and test, at introductory, interview and competition difficulty, drawn from open-access coding challenge sites and shipped with 131,777 test cases and Python solutions. Published at NeurIPS 2021, it spans the gap between function-completion tasks and full competitive programming.

Verified 2026-08-13 via the codeparrot/apps dataset card on Hugging Face.

Openness

5 high confidence
5.0
data
downloadable(HF parquet, codeparrot/apps)
license
MIT(OSI/open)
datasheet
present(card documents 10k problems, 131,777 tests, difficulty levels, schema)
redistributable
yes
paper
public(NeurIPS 2021)

Clean open data: MIT-licensed, downloadable, with a documented datasheet and a public paper. Fully open.

  • https://huggingface.co/datasets/codeparrot/apps recorded 2026-08-13

    `license:mit` in the repo tags; the embedded repo state reads `"gated":false`, so the parquet downloads without a barrier; the dataset card documents the 10,000 problems and difficulty levels; 17,169 downloads in the trailing 30 days

Adoption

3 medium confidence
3.0

17,673 Hugging Face downloads in the trailing 30 days for codeparrot/apps, which puts it in the 10K-100K band, level 3 on the dataset adoption scale. That scale tops out at >1M, an order of magnitude below the ones used for software and models, because no dataset in this corpus has ever passed 10M downloads. APPS is a well-established code-generation benchmark, cited for years but partly superseded by harder contamination-free suites such as LiveCodeBench and SWE-bench.

Capability

not assessed

A dataset is not capability-scored, so this axis is left unscored. The page publishes a static problem set with no performance or feature claim for the capability axis to read.

Verified 2026-08-12