Back to Gap Map Model components / Benchmark / eval datasets

APPS

open / Overall score: 3.0

APPS is a benchmark for Python code generation: 10,000 coding problems split evenly between train and test, at introductory, interview and competition difficulty, drawn from open-access coding challenge sites and shipped with 131,777 test cases and Python solutions. Published at NeurIPS 2021, it ranges from simple one-line solutions to substantial algorithmic challenges.

Openness

5 high confidence

5.0

data: downloadable(HF parquet, codeparrot/apps)
license: MIT(OSI/open)
datasheet: present(card documents 10k problems, 131,777 tests, difficulty levels, schema)
redistributable: yes
paper: public(NeurIPS 2021)

Clean open data: MIT-licensed, downloadable, with a documented datasheet and a public paper. Fully open.

https://huggingface.co/datasets/codeparrot/apps recorded 2026-09-29
Dataset page carries the license tag "license:mit" and is not gated; the description reads "APPS is a benchmark for Python code generation, it includes 10,000 problems", and the card documents "131777 test cases", the difficulty levels "Introductory, Interview and Competition" and a Data Fields table for the schema.

Adoption

3 medium confidence

3.0

Adoption is measured by Hugging Face downloads of codeparrot/apps, the dataset's only declared artifact. APPS remains a long-cited coding benchmark, though newer, harder suites such as LiveCodeBench and SWE-bench have taken over much of that role.

https://huggingface.co/api/datasets/codeparrot/apps recorded 2026-08-12
17,673 downloads in the trailing 30 days for codeparrot/apps

Capability

not assessed

A dataset is not capability-scored, so capability is left unscored. The page publishes a static problem set with no performance or feature claim to score.

https://huggingface.co/datasets/codeparrot/apps recorded 2026-09-29
Dataset card describes the corpus as "APPS is a benchmark for Python code generation, it includes 10,000 problems" curated from "open-access sites where programmers share problems with each other", with "131777 test cases"; searched the body for leaderboard, accuracy, SOTA, throughput and latency claims: none.

Verified 2026-09-27