LiveBench

questions

public

answers

public(ground_truth released)

license

Apache-2.0/MIT(repo LICENSE is Apache-2.0, carried over from FastChat

datasheet

present(docs/DATASHEET.md, a full datasheets-for-datasets questionnaire covering motivation, composition, collection and distribution)

access

ungated(HF datasets download without a gate)

All questions, answers, and code are released openly; rotation is for freshness, not access control. The documentation is a full datasheets-for-datasets questionnaire in the repository rather than a Hugging Face card, and it states the distribution license outright.

• https://huggingface.co/datasets/livebench/reasoning recorded 2026-06-17

  public ground_truth column, release/removal dates

• https://huggingface.co/datasets/livebench/reasoning/raw/main/README.md recorded 2026-10-13

  card describing the benchmark and linking the paper, leaderboard and datasheet; the declared features include a `ground_truth` string column alongside `turns`, so the answers ship with the questions; no gate and no license field in the card metadata

• https://raw.githubusercontent.com/LiveBench/LiveBench/main/docs/DATASHEET.md recorded 2026-10-13

  the datasheets-for-datasets questionnaire, covering motivation, composition, collection, preprocessing, uses, distribution and maintenance; under "When will the dataset be released/first distributed? What license (if any) is it distributed under?": "The benchmark suite is public as of June 12, 2024, distributed under the Apache License 2.0."

• https://raw.githubusercontent.com/LiveBench/LiveBench/main/LICENSE recorded 2026-10-13

  Apache License 2.0, prefaced "The original LICENSE from https://github.com/lm-sys/FastChat is copied below"

Adoption is measured as Hugging Face downloads summed across LiveBench's ten per-domain dataset repositories, since the benchmark ships as a family of datasets rather than a single artifact. Most of the total comes from the six main task domains — math, reasoning, model judgment, coding, data analysis, instruction following and language — with three smaller companion sets contributing the rest.

• https://huggingface.co/api/datasets/livebench/coding recorded 2026-08-12

  4,761 downloads in the trailing 30 days for livebench/coding

• https://huggingface.co/api/datasets/livebench/data_analysis recorded 2026-08-12

  4,270 downloads in the trailing 30 days for livebench/data_analysis

• https://huggingface.co/api/datasets/livebench/instruction_following recorded 2026-08-12

  4,250 downloads in the trailing 30 days for livebench/instruction_following

• https://huggingface.co/api/datasets/livebench/language recorded 2026-08-12

  4,165 downloads in the trailing 30 days for livebench/language

• https://huggingface.co/api/datasets/livebench/liveswebench recorded 2026-08-12

  370 downloads in the trailing 30 days for livebench/liveswebench

• https://huggingface.co/api/datasets/livebench/liveswebench-patches recorded 2026-08-12

  145 downloads in the trailing 30 days for livebench/liveswebench-patches

• https://huggingface.co/api/datasets/livebench/math recorded 2026-08-12

  6,923 downloads in the trailing 30 days for livebench/math

• https://huggingface.co/api/datasets/livebench/model_answer recorded 2026-08-12

  150 downloads in the trailing 30 days for livebench/model_answer

• https://huggingface.co/api/datasets/livebench/model_judgment recorded 2026-08-12

  5,365 downloads in the trailing 30 days for livebench/model_judgment

• https://huggingface.co/api/datasets/livebench/reasoning recorded 2026-08-12

  5,633 downloads in the trailing 30 days for livebench/reasoning

A dataset is not 'capable' the way software or a model is, so capability is left unscored. LiveBench grades every question against objective ground truth rather than an LLM judge and rotates its six task domains monthly to resist contamination.

• https://arxiv.org/abs/2406.19314 recorded 2026-09-29

  The arXiv abstract page for the LiveBench paper says "We release all questions, code, and model answers. Questions are added and updated on a monthly basis" and that top models score "below 70% accuracy". Searched the body for a capability score, SOTA and usage counts: none.