LiveCodeBench
LiveCodeBench TeamContinuously updated coding benchmark that collects fresh problems from LeetCode, Codeforces and AtCoder after a model's training cutoff, so evaluation stays contamination-free. It covers code generation, self-repair, execution and test-output prediction, and ships release-versioned snapshots with hidden test cases bundled.
The dataset card declares a bare Creative Commons tag naming no variant, and the harness repository's terms make no statement about the problems, which the openness axis resolves. Verified 2026-08-13 via the livecodebench/code_generation_lite card, its raw README and the harness repository.
Openness
3 medium confidence- data
- downloadable(HF parquet)
- license
- cc(the card YAML names the Creative Commons family and no variant
- datasheet
- present(README documents versions/sources/structure)
- redistributable
- yes
- splits
- public problems with hidden test cases bundled
Downloadable with a datasheet, but the only license statement the project makes about the corpus is a bare "cc" naming no variant, and CC-BY and CC-BY-NC sit two tiers apart. No variant is stated anywhere the project publishes: not in the card front matter, the Hub API, the sibling code_generation and test_generation corpora, the repository LICENSE (MIT, which covers the lcb_runner harness rather than the problems), the README, ERRATA.md, the project site, the leaderboard or the paper. Files that download freely without a grant to use them are not open, and not closed either, so this sits at gated - the barrier is the missing permission rather than a login. The problems are collected from LeetCode, AtCoder and Codeforces, which is a plausible reason no variant has been committed to.
- https://huggingface.co/datasets/livecodebench/code_generation_lite recorded 2026-08-13
`license:cc` in the repo tags with no variant; the embedded repo state reads `"gated":false`, so the parquet downloads freely; the README/datasheet still renders the release_v1..v5 change log; 93,572 downloads in the trailing 30 days
- https://huggingface.co/datasets/livecodebench/code_generation_lite/blob/main/README.md recorded 2026-06-04
YAML metadata license: cc
- https://huggingface.co/datasets/livecodebench/code_generation_lite/raw/main/README.md recorded 2026-08-13
card front matter is `license: cc` with tags, pretty_name and size_categories, and nothing naming a Creative Commons variant; the body is a change log of release_v1..v5, a dataset description and usage instructions
- https://huggingface.co/api/datasets/livecodebench/code_generation_lite recorded 2026-08-13
cardData.license is the string "cc" and the repository tags carry `license:cc`; the sibling corpora code_generation and test_generation carry the same bare `cc`, and execution carries no license at all. The open discussions on the repo are about loading, parquet conversion and version sizes - none asks or answers what the CC variant is
- https://raw.githubusercontent.com/LiveCodeBench/LiveCodeBench/main/LICENSE recorded 2026-08-13
MIT License, copyright 2024 LiveCodeBench. This is the repository holding lcb_runner, the evaluation harness; the problems are distributed from the Hugging Face repos and this file makes no statement about them
- https://raw.githubusercontent.com/LiveCodeBench/LiveCodeBench/main/README.md recorded 2026-08-13
217 lines covering install, running the harness, scenarios and submissions, with no licensing or terms section at all
Adoption
3 medium confidence91,719 Hugging Face downloads in the trailing 30 days for livecodebench/code_generation_lite, the single declared artifact, which puts it in the 10K-100K band, level 3 on the dataset adoption scale. That scale tops out at >1M.
- https://huggingface.co/api/datasets/livecodebench/code_generation_lite recorded 2026-08-13
91,719 downloads in the trailing 30 days for livecodebench/code_generation_lite
Capability
not assessedA dataset is not 'capable', so this axis is left unscored. Its contamination resistance - the "live", continuously updated design - is a real quality strength, but openness and adoption are what carry this category.
- https://huggingface.co/datasets/livecodebench/code_generation_lite recorded 2026-08-13
a static release-versioned problem corpus with hidden test cases bundled; no performance, throughput or feature claim on the page for the capability axis to read
Verified 2026-08-13