Back to Gap Map Model components / Benchmark / eval datasets

MultiPL-E

Northeastern University / Brown / Wellesley
restricted / Overall score: 3.0

Translation of HumanEval's 161 problems and MBPP's 397 into 22 to 23 programming languages including Ada, C++, C#, Go, Haskell, Java, JavaScript, Lua, OCaml, PHP, R, Ruby, Rust, Scala, Shell, Swift and TypeScript. It lets code-generation performance be compared across languages rather than in Python alone, and is integrated into the BigCode evaluation harness.

Openness

3 medium confidence
3.0
data
downloadable(HF parquet, 47 subsets)
license
BSD-3-Clause-with-ML-Restriction(the nuprl/MultiPL-E repository LICENSE, whose clause 4 forbids using the contents as training data for any machine learning model
datasheet
present(card documents prompts/doctests/tests/stop-tokens)
redistributable
yes(HF)

MultiPL-E's GitHub repository ships under a modified BSD 3-Clause license whose fourth clause forbids using the contents as training data for any machine-learning model; where the Hugging Face copy's plain MIT tag disagrees, the repository's own license governs. The dataset downloads freely and may be used commercially, but not to train a model. A license that allows commercial use while barring one class of use is bounded, so its tier is use_bounded.

Adoption

3 high confidence
3.0

Adoption is measured by Hugging Face downloads of nuprl/MultiPL-E, the dataset's only declared artifact.

Capability

not assessed

MultiPL-E is a dataset and translation framework rather than a system with capabilities to measure, so capability is left unscored.

  • https://huggingface.co/datasets/nuprl/MultiPL-E recorded 2026-09-29

    Hugging Face dataset card stating "MultiPL-E is a dataset for evaluating large language models for code" across 22 programming languages, i.e. a static set of translated prompts and tests; searched the body for SOTA, throughput and pass@1: none.

Verified 2026-09-27