Model components / Benchmark / eval datasets

CodeContests

A competitive programming dataset containing ~13,000 problems collected from Codeforces, CodeChef, and other contest platforms, each with problem descriptions, solutions in multiple languages, and test cases. Introduced in the AlphaCode paper (2022). Designed to test models on problems requiring algorithmic reasoning far beyond typical function-completion benchmarks, the hard end of code generation evaluation.

CodeContests (deepmind/code_contests), competitive-programming dataset built for AlphaCode (arXiv:2203.07814). 3,760 train / 117 valid / 165 test problems (4,044 total per HF viewer), ~7.6 GB, with public/private/generated test cases and correct+incorrect human solutions in multiple languages. Verified live on HF June 2026 (~56k downloads last month). [Verifier note: original record stated 13,328 train; HF dataset viewer shows 3,760 train / 4,044 total; corrected.]

Openness

5 high confidence

5.0

license: CC-BY-4.0(open data license, attribution-only)
redistributability: full(downloadable, ungated)
datasheet: yes(comprehensive dataset card + arXiv:2203.07814)
size: ~4k problems w/ rich test cases
sources: Codeforces/AtCoder/CodeChef/Aizu/HackerEarth + Description2Code(MIT)+CodeNet(Apache-2.0)

Fully open: CC-BY-4.0, ungated, downloadable, with a thorough datasheet documenting provenance. Clean open-data case on the dataset-openness path.

https://huggingface.co/datasets/deepmind/code_contests recorded 2026-06-04
CC-BY-4.0 license; not gated; 3,760 train/117 val/165 test (4,044 total); dataset card present; ~56k downloads last month

Adoption

3 medium confidence

3.0

Well-known competitive-programming benchmark (the AlphaCode dataset) and a recurring reference for code-reasoning evals, but materially less ubiquitous than HumanEval/GSM8K/SWE-bench in current release reports. ~56k HF downloads/month (lowest in the batch) places it in the 100K-1M annualized band. Honest signal: solid research adoption, not headline-eval scale.

https://huggingface.co/datasets/deepmind/code_contests recorded 2026-06-04
~56k downloads last month

Capability

not assessed

Dataset, not 'capable'. High difficulty / good discriminating power for code reasoning, but per recipe capability is n/a.

Unchanged since 2026-06-24 (last edited, not re-checked)