RewardBench
Allen Institute for AIBenchmark for evaluating reward models, pairing prompts with chosen and rejected responses across chat, reasoning and safety subsets. RewardBench 2 extends it with harder, unseen human prompts, and both versions publish their datasets, results and a public leaderboard. The repository provides a common inference format for scoring most open reward models.
One product covering RewardBench and RewardBench 2; the repo README fronts both with datasets, results and leaderboard links. PyPI rewardbench's home_page points at a 404 path, so no pypi artifact is declared.
Openness
5 high confidence- license
- ODC-BY(license odc-by on both Hub dataset cards
- access
- open(both Hub datasets ungated ("gated":false), downloadable parquet)
- answers
- released(chosen/rejected pairs are the labels and ship in the public splits
- datasheet
- present(dataset cards on both Hub releases with schema, subsets and citation)
Both RewardBench datasets are released under ODC-BY and download without a gate, and each carries a complete dataset card. RewardBench 2 is the current release and carries the same license as the original, so nothing changes with the newer version.
- https://huggingface.co/api/datasets/allenai/reward-bench recorded 2026-10-07
Dataset API for allenai/reward-bench: `gated: false`, `private: false`, cardData.license `odc-by`, with the filtered and raw parquet splits and the build scripts listed.
- https://huggingface.co/api/datasets/allenai/reward-bench-2 recorded 2026-10-07
Dataset API for allenai/reward-bench-2: `gated: false`, `private: false`, cardData.license `odc-by`, with the test parquet split listed.
- https://huggingface.co/datasets/allenai/reward-bench/raw/main/README.md recorded 2026-10-07
card front matter license: odc-by; features prompt/chosen/chosen_model/rejected/ rejected_model; subsets and filtering documented
- https://raw.githubusercontent.com/allenai/reward-bench/main/LICENSE recorded 2026-10-07
Apache License 2.0 full body, covering the evaluation code
- https://raw.githubusercontent.com/allenai/reward-bench/main/README.md recorded 2026-10-07
fronts RewardBench and RewardBench 2 together — leaderboard space, both eval datasets, both results sets, papers arXiv:2403.13787 and arXiv:2506.01937
Adoption
3 high confidenceAdoption is measured by Hugging Face downloads summed across the two declared datasets, allenai/reward-bench and allenai/reward-bench-2. RewardBench is a widely used reward-model leaderboard, but this reading tracks the measured download count rather than that reputation.
- https://huggingface.co/api/datasets/allenai/reward-bench recorded 2026-09-01
10,022 downloads in the trailing 30 days
- https://huggingface.co/api/datasets/allenai/reward-bench-2 recorded 2026-09-01
4,212 downloads in the trailing 30 days
Capability
not assessedA dataset is not 'capable', so capability is left unscored, mirroring the category pattern (gsm8k).
- https://huggingface.co/datasets/allenai/reward-bench/raw/main/README.md recorded 2026-10-07
a static preference-pair corpus; no performance, throughput or feature claim for the capability axis to read
Verified 2026-09-27