AI Potluck
Contribute
Back to Gap Map Model components / Benchmark / eval datasets

HellaSwag

Allen Institute for AI
open / Overall score: 4.0(strong)

Commonsense natural-language-inference and sentence-completion benchmark from 2019, holding 59,950 four-way multiple-choice examples over contexts derived from ActivityNet. It tests whether a model picks the plausible continuation of an everyday situation, and is reported in most language-model releases.

The registry attributes the dataset to the Allen Institute; the primary authors are at the University of Washington and AI2.

Openness

5 high confidence
5.0
data
open(59,950 examples, downloadable on HF)
license
MIT(permissive, redistributable)
datasheet
yes(HF dataset card + ACL 2019 paper)
access
public(train/val/test all released, not held-out)

Fully open under MIT with a dataset card - the canonical open-data benchmark profile. The repository carries no license tag, so the MIT is established by the body of the card rather than by a tag.

  • https://huggingface.co/datasets/Rowan/hellaswag recorded 2026-08-13

    the card's Licensing Information section reads "MIT" with a link to github.com/rowanz/hellaswag/blob/master/LICENSE; the embedded repo state reads `"gated":false` and the tags array carries no license tag; train/validation/test all public; 307,689 downloads in the trailing 30 days

Adoption

4 high confidence
4.0

Adoption is measured by Hugging Face downloads of Rowan/hellaswag. HellaSwag is a commonsense-reasoning benchmark cited in most language-model release reports and included on the Hugging Face Open LLM Leaderboard.

Capability

not assessed

Capability is not a meaningful axis for an eval dataset, so it is left unscored.

  • https://huggingface.co/datasets/Rowan/hellaswag recorded 2026-09-29

    The dataset card describes "HellaSwag: Can a Machine Really Finish Your Sentence? is a new dataset for commonsense NLI"; it lists 59,950 rows in parquet format. Searched the body for performance, throughput and latency claims: none.

Verified 2026-09-27, except some scores, still awaiting re-confirmation