GPQA

license

CC-BY-4.0(open,redistributable+attribution)

access

gated(HF login + click-through agreement NOT to publish examples in plaintext, to limit training-corpus contamination)

datasheet

present(paper+card)

splits

public(no hidden held-out test)

GPQA's CC-BY-4.0 license is fully open, but access requires a Hugging Face login and a click-through agreement not to publish the questions in plaintext, so the corpus reads as gated rather than open. The gate exists to limit training-corpus contamination, a more sympathetic reason than most, but that does not exempt GPQA from being treated like any other gated dataset.

• https://arxiv.org/abs/2311.12022 recorded 2026-09-29

  The abstract reads "a challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry".

• https://huggingface.co/datasets/Idavidrein/gpqa recorded 2026-09-29

  The page tags the dataset "license:cc-by-4.0", lists the repo state as "gated":"auto", says "You need to agree to share your contact information to access this dataset", and the gate prompt reads "You agree to NOT reveal examples from this dataset in plain text or images online, to reduce the risk of leakage into foundation model training corpora"; a dataset card titled "Dataset Card for GPQA" renders.

Adoption is measured through Hugging Face downloads across GPQA's declared dataset artifacts. GPQA Diamond is a standard reasoning benchmark cited across frontier model release reports, and it underlies capability scores for flagship models throughout this map.

• https://huggingface.co/api/datasets/Idavidrein/gpqa recorded 2026-08-13

  110,577 downloads in the trailing 30 days for Idavidrein/gpqa

A benchmark dataset is not 'capable', so capability is left unscored; openness and adoption carry this category.

• https://huggingface.co/datasets/Idavidrein/gpqa recorded 2026-09-29

  The page describes "GPQA is a multiple-choice, Q&A dataset of very hard questions written and validated by experts in biology, physics, and chemistry", a static question corpus; searched the body for throughput, latency and SOTA: none, and the Hub-generated model leaderboard widget is platform listing, not a performance claim by the dataset.