AI Potluck
Model components / Benchmark / eval datasets

MBPP

Google Research

Mostly Basic Python Problems: 974 crowd-sourced Python tasks solvable by an entry-level programmer, each with a task description, a reference solution and three automated test cases, plus a 427-problem hand-verified subset. Where HumanEval tests function-level generation from a detailed docstring, MBPP tests natural-language-to-code translation across a broader difficulty range.

Verified 2026-08-13 via the google-research-datasets/mbpp dataset card on Hugging Face.

Openness

5 high confidence
5.0
license
CC-BY-4.0(open data license)
redistributability
full(downloadable, ungated)
datasheet
yes(extensive dataset card)
size
~974 full / 427 sanitized problems w/ test_list
splits
train/test/validation/prompt public

Fully open: CC-BY-4.0, ungated, downloadable, with an extensive dataset card. Clean open-data case; the standard companion to HumanEval for basic Python code-gen eval.

Adoption

4 medium confidence
4.0

208,317 Hugging Face downloads in the trailing 30 days for google-research-datasets/mbpp, which puts it in the 100K-1M band, level 4 on the dataset adoption scale. That scale tops out at >1M, an order of magnitude below the ones used for software and models, because no dataset in this corpus has ever passed 10M downloads. MBPP is a standard basic-Python code-generation benchmark, routinely paired with HumanEval in code-model release reports.

Capability

not assessed

A dataset is not 'capable', so this axis is left unscored. MBPP is largely saturated by frontier models, but that is a quality caveat rather than a capability score.

Verified 2026-08-12