LiveBench

questions

public

answers

public(ground_truth released)

license

Apache-2.0/MIT(repo LICENSE is Apache-2.0, carried over from FastChat

datasheet

present(docs/DATASHEET.md, a full datasheets-for-datasets questionnaire covering motivation, composition, collection and distribution)

access

ungated(HF datasets download without a gate)

All questions, answers, and code are released openly; rotation is for freshness, not access control. The documentation is a full datasheets-for-datasets questionnaire in the repository rather than a Hugging Face card, and it states the distribution license outright.

• https://huggingface.co/datasets/livebench/reasoning recorded 2026-06-17

  public ground_truth column, release/removal dates

• https://huggingface.co/datasets/livebench/reasoning/raw/main/README.md recorded 2026-08-12

  card describing the benchmark and linking the paper, leaderboard and datasheet; the declared features include a `ground_truth` string column alongside `turns`, so the answers ship with the questions; no gate and no license field in the card metadata

• https://raw.githubusercontent.com/LiveBench/LiveBench/main/docs/DATASHEET.md recorded 2026-08-12

  the datasheets-for-datasets questionnaire, covering motivation, composition, collection, preprocessing, uses, distribution and maintenance; under "When will the dataset be released/first distributed? What license (if any) is it distributed under?": "The benchmark suite is public as of June 12, 2024, distributed under the Apache License 2.0."

• https://raw.githubusercontent.com/LiveBench/LiveBench/main/LICENSE recorded 2026-08-12

  Apache License 2.0, prefaced "The original LICENSE from https://github.com/lm-sys/FastChat is copied below"

36,032 Hugging Face downloads in the trailing 30 days, summed across the ten LiveBench category corpora: math 6,923; reasoning 5,633; model_judgment 5,365; coding 4,761; data_analysis 4,270; instruction_following 4,250; language 4,165; liveswebench 370; model_answer 150; and liveswebench-patches 145. That puts it in the 10K-100K band, level 3 on the dataset adoption scale.

• https://huggingface.co/api/datasets/livebench/math recorded 2026-08-12

  6,923 downloads in the trailing 30 days for livebench/math

• https://huggingface.co/api/datasets/livebench/reasoning recorded 2026-08-12

  5,633 downloads in the trailing 30 days for livebench/reasoning

• https://huggingface.co/api/datasets/livebench/model_judgment recorded 2026-08-12

  5,365 downloads in the trailing 30 days for livebench/model_judgment

• https://huggingface.co/api/datasets/livebench/coding recorded 2026-08-12

  4,761 downloads in the trailing 30 days for livebench/coding

• https://huggingface.co/api/datasets/livebench/data_analysis recorded 2026-08-12

  4,270 downloads in the trailing 30 days for livebench/data_analysis

• https://huggingface.co/api/datasets/livebench/instruction_following recorded 2026-08-12

  4,250 downloads in the trailing 30 days for livebench/instruction_following

• https://huggingface.co/api/datasets/livebench/language recorded 2026-08-12

  4,165 downloads in the trailing 30 days for livebench/language

• https://huggingface.co/api/datasets/livebench/liveswebench recorded 2026-08-12

  370 downloads in the trailing 30 days for livebench/liveswebench

• https://huggingface.co/api/datasets/livebench/model_answer recorded 2026-08-12

  150 downloads in the trailing 30 days for livebench/model_answer

• https://huggingface.co/api/datasets/livebench/liveswebench-patches recorded 2026-08-12

  145 downloads in the trailing 30 days for livebench/liveswebench-patches

Among the most influential contamination-resistant benchmarks: broader than the code-only LiveCodeBench, and judge-free, since every question is auto-scored against objective ground truth.

• https://arxiv.org/abs/2406.19314 recorded 2026-08-13

  "LiveBench - A Challenging, Contamination-Limited LLM Benchmark"; the abstract still lists the three release properties, names the six task domains, reports top models below 70% accuracy, and says "We release all questions, code, and model answers. Questions are added and updated on a monthly basis"