Back to Gap Map Model components / Benchmark / eval datasets

CodeContests

open / Overall score: 3.0

Competitive-programming dataset built for the AlphaCode paper, holding 4,044 problems collected from Codeforces, CodeChef and other contest platforms, each with a description, correct and incorrect human solutions in several languages, and public, private and generated test cases.

An earlier version of this record stated 13,328 training problems; the dataset viewer shows 3,760 train and 4,044 total, and the record was corrected.

Openness

5 high confidence

5.0

license: CC-BY-4.0(open data license, attribution-only)
redistributability: full(downloadable, ungated)
datasheet: yes(comprehensive dataset card + arXiv:2203.07814)
size: ~4k problems w/ rich test cases
sources: Codeforces/AtCoder/CodeChef/Aizu/HackerEarth + Description2Code(MIT)+CodeNet(Apache-2.0)

Fully open: CC-BY-4.0, ungated, downloadable, with a thorough datasheet documenting provenance. A clean open-data case.

https://huggingface.co/datasets/deepmind/code_contests recorded 2026-09-29
Dataset page shows "license:cc-by-4.0" in its tags, a card titled "Dataset Card for CodeContests" and "It consists of programming problems, from a variety of sources"; searched the body for non-commercial and holdout terms: none.

Adoption

3 medium confidence

3.0

Adoption is measured through Hugging Face downloads of deepmind/code_contests. This is the AlphaCode competitive-programming dataset, a recurring reference for code-reasoning evaluation, though it appears less often in current release reports than HumanEval or GSM8K, and this category counts dataset downloads on a far smaller scale than it does for software or models.

https://huggingface.co/api/datasets/deepmind/code_contests recorded 2026-08-12
50,563 downloads in the trailing 30 days for deepmind/code_contests

Capability

not assessed

A dataset is not 'capable', so capability is left unscored. CodeContests is hard and discriminates well on code reasoning, but that is a quality strength rather than a capability score.

https://huggingface.co/datasets/deepmind/code_contests recorded 2026-09-29
Dataset card opens "CodeContests is a competitive programming dataset for machine-learning" and says "The leaderboard for this task is available", linking to an external paperswithcode leaderboard; the page states no accuracy figure or performance claim for the corpus itself.

Verified 2026-09-27