Princeton NLP / OpenAI

license

undeclared on HF dataset card (underlying SWE-bench harness repo is MIT)

redistributability

full(downloadable parquet, ungated)

datasheet

yes(dataset card + arXiv:2310.06770)

held-out

no hidden split in Verified itself (the public 500-instance set is fully released

data-source

real GitHub PR/issue pairs

Effectively open (ungated, downloadable, full gold patches and tests, datasheet present) but the HF dataset card declares NO explicit license field, so the data-license class is inferred from the MIT harness repo and public redistribution rather than a stated data license. Scored 4 not 5 on that ambiguity. (Distinct from SWE-bench's broader/hidden variants which can carry held-out splits.)

• https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified recorded 2026-06-04

  500 instances, ~2.1 MB, not gated, dataset card present, gold patch + FAIL_TO_PASS/PASS_TO_PASS tests included, leaderboard swebench.com

• https://github.com/SWE-bench/SWE-bench/blob/main/LICENSE recorded 2026-06-04

  MIT License for the SWE-bench harness

• https://huggingface.co/datasets/princeton-nlp/SWE-bench_Verified/blob/main/README.md recorded 2026-06-04

  dataset card metadata has no license field declared