UltraFeedback
OpenBMBGPT-4-annotated preference dataset of roughly 64,000 prompts, 256,000 responses and 380,000 fine-grained feedback annotations across instruction-following, truthfulness, honesty and helpfulness. Prompts are drawn from six public datasets with responses from 17 different models. Built by OpenBMB, it is used to train reward and critic models in open RLHF and DPO pipelines.
Openness
5 high confidence- license
- mit
- card
- present
- ungated
- yes
- rows
- 63,967
- size
- 940MB
Publicly downloadable and ungated under the permissive MIT license.
- https://huggingface.co/api/datasets/openbmb/UltraFeedback recorded 2026-09-29
The Hub API JSON marks the dataset public and not gated (a false gated flag) and carries the tag "license:mit" beside a description reading "large-scale, fine-grained, diverse preference dataset".
- https://huggingface.co/datasets/openbmb/UltraFeedback recorded 2026-09-29
The dataset page renders the card text "UltraFeedback is a large-scale, fine-grained, diverse preference dataset, used for training powerful reward models and critic models" and the sentence "We sample 63,967 instructions from 6 public available".
- https://huggingface.co/datasets/openbmb/UltraFeedback/raw/main/README.md recorded 2026-09-29
The raw card has front matter "license: mit" and says "UltraFeedback is a **large-scale, fine-grained, diverse preference dataset**, used for training powerful reward models and critic models", with sections on instruction, model and principle sampling; "We sample 63,967 instructions from 6 public available and high-quality datasets".
Adoption
2 high confidenceAdoption is measured by Hugging Face downloads of openbmb/UltraFeedback, the only usage channel on record for UltraFeedback.
- https://huggingface.co/api/datasets/openbmb/UltraFeedback recorded 2026-08-13
5,456 downloads in the trailing 30 days for openbmb/UltraFeedback
Capability
4 high confidenceIn the Zephyr ablation, DPO on UltraFeedback lifts MT-Bench from 6.64 to 7.00, and the final Zephyr model reaches 7.34. Its annotation pipeline is the template for the Tulu 3 and OLMo 2 preference data, and a decontaminated subset appears in Tulu 3's preference mixture.
- https://ar5iv.labs.arxiv.org/html/2310.16944 recorded 2026-09-29
Zephyr paper (ar5iv) Table 3 is titled "Ablation of different alignment methods on the base Mistral 7B model" with MT-Bench cells "6.64" for SFT only and "7.00" for DPO plus SFT, Table 2 lists Zephyr 7B at "7.34", and the text says "fine-tunes the base model directly with DPO for one epoch on UltraFeedback".
- https://arxiv.org/html/2411.15124 recorded 2026-09-29
Tulu 3 paper HTML says "We also include a decontaminated subset of UltraFeedback" in the SFT mixture and builds preferences "by adapting and advancing the UltraFeedback pipeline".
- https://arxiv.org/html/2501.00656 recorded 2026-09-29
OLMo 2 paper HTML says "The core strategy of the Tülu 3 pipeline for PreFT is building upon and scaling the UltraFeedback pipeline".
Verified 2026-09-29, except some scores, still awaiting re-confirmation