TRL

license

Apache-2.0(OSI)

source

public(huggingface/trl)

managed-tier

none

core features withheld

no

Apache-2.0 throughout, an OSI license, with the full source public. There is no proprietary core and no paid managed-training service that withholds features from the published package.

• https://github.com/huggingface/trl/blob/main/LICENSE recorded 2026-08-08

  Rendered blob of the repository LICENSE. Body carries the full Apache License text: the heading block "Apache License / Version 2.0, January 2004 / http://www.apache.org/licenses/", the section "TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION" with "1. Definitions." following it, and the appendix notice "Copyright 2020-2026 The HuggingFace Team". No added use restriction or field-of-use clause.

• https://huggingface.co/docs/trl/index recorded 2026-09-23

  The "Taxonomy" section prints the complete trainer set of the installed library - online: GRPOTrainer, RLOOTrainer, OnlineDPOTrainer, NashMDTrainer, PPOTrainer, XPOTrainer; reward modeling: RewardTrainer, PRMTrainer; offline: SFTTrainer, DPOTrainer, KTOTrainer, BCOTrainer, CPOTrainer, ORPOTrainer; knowledge distillation: GKDTrainer, MiniLLMTrainer. The only per-trainer markers are the legend's own two (vLLM support and experimental); no trainer is marked as belonging to a paid or hosted tier.

• https://huggingface.co/pricing recorded 2026-09-23

  Hugging Face's paid offering, read for the core-gated question. The plans are PRO ($9/month), Team ($20/month per user) and Enterprise ($50/month per user), plus per-TB Hub storage ($8-18/TB/mo) and hourly Spaces hardware. Every listed benefit is Hub/account scoped - private storage, inference credits, ZeroGPU quota, SSO, audit logs, resource groups. The page does not mention TRL at all (zero occurrences of the string in the body) and gates no TRL trainer or feature behind a plan.

• https://pypi.org/project/trl/ recorded 2026-09-23

  Package page for trl. "License expression: Apache-2.0" with a link to the SPDX License List; Author: Leandro von Werra; "Requires Python >=3.10"; "Provides Extra: bco, deepspeed, kernels, liger, peft, quality, quantization, scikit, test, vllm, vlm, math" - no commercial or enterprise extra. Confirms the distributed wheel carries the same Apache-2.0 terms as the repository.

• https://raw.githubusercontent.com/huggingface/trl/main/LICENSE recorded 2026-09-23

  11,355-byte plain-text Apache License Version 2.0 (January 2004), 201 lines, unmodified through section 9 and the appendix, closing with "Copyright 2020-2026 The HuggingFace Team" and the standard "Licensed under the Apache License, Version 2.0" notice. Stable plain-text form of the same file the cited blob URL renders.

• https://raw.githubusercontent.com/huggingface/trl/main/pyproject.toml recorded 2026-09-23

  [project] declares name = "trl", license = "Apache-2.0", license-files = ["LICENSE"], requires-python = ">=3.10", and [project.scripts] trl = "trl.cli:main" - the CLI entry point ships in the same package. [project.optional-dependencies] lists bco, deepspeed, kernels, liger and further extras, all pulling open packages (scikit-learn, deepspeed, liger-kernel, transformers); there is no commercial, enterprise or premium extra and no reference to a separate closed distribution.

• https://raw.githubusercontent.com/huggingface/trl/main/README.md recorded 2026-09-23

  Installation section gives three routes to the running product from the published repo: "pip install trl", "pip install git+https://github.com/huggingface/trl.git", and "git clone https://github.com/huggingface/trl.git; cd trl/; pip install -e .[dev]". Highlights name SFTTrainer, GRPOTrainer, DPOTrainer and KTOTrainer, Accelerate scaling "from single GPU to multi-node clusters" via DDP and DeepSpeed, and PEFT LoRA/QLoRA. The CLI section shows "trl sft", "trl dpo", "trl kto" invocations. The file offers no paid, premium or enterprise edition and names no feature that the published source cannot build.

TRL is the standard library for RLHF and post-training in the Hugging Face stack, integrated with Transformers and Accelerate and used across research labs and open-model post-training pipelines. Its PyPI download volume each month is what the reading measures, reflecting how central it has become to that ecosystem.

• https://pypistats.org/api/packages/trl/recent recorded 2026-08-08

  Full JSON body: {"data":{"last_day":139036,"last_month":3677381,"last_week":884630},"package":"trl","type":"recent_downloads"}. Machine-readable form of the same figure the HTML page renders, so the reading is re-derivable without parsing HTML.

• https://pypistats.org/packages/trl recorded 2026-08-08

  Package page for trl. "Downloads last day: 139,036 / Downloads last week: 884,630 / Downloads last month: 3,677,381". Header block reads "Latest version: 1.9.2" and "Summary: Train transformer language models with reinforcement learning."

TRL covers a broad set of post-training methods: supervised fine-tuning, DPO, KTO and ORPO preference training, GRPO and other online methods, PPO, and reward modeling. It hands large-scale distributed training to Hugging Face Accelerate and DeepSpeed rather than implementing its own parallelism, which leaves it a tier behind Megatron-LM.

• https://huggingface.co/docs/trl/distributing_training recorded 2026-09-28

  The distributed-training guide has a "Multi-Node Training" section with an "Accelerate Configuration" subsection and says to launch distributed training "by running" after "accelerate config", so scale-out goes through Accelerate.

• https://huggingface.co/docs/trl/dpo_trainer recorded 2026-09-28

  The DPO trainer page documents preference-loss variants and says "TRL exposes this as <code>loss_type="ipo"</code>" for the IPO variant.

• https://huggingface.co/docs/trl/index recorded 2026-09-28

  The stable docs index opens its trainer overview with "Below is an overview of TRL trainers, organized by maturity and method type" and lists GRPOTrainer, RLOOTrainer, RewardTrainer, SFTTrainer, DPOTrainer, KTOTrainer and DistillationTrainer as stable, with OnlineDPOTrainer, CPOTrainer, ORPOTrainer and GKDTrainer under Experimental. Searched the body for PPOTrainer, FP8 and BF16: none.

• https://huggingface.co/docs/trl/main/en/ppo_trainer recorded 2026-09-28

  The main-branch PPO page describes "Trainer for Proximal Policy Optimization (PPO)" and documents the class at "trl.experimental.ppo.PPOTrainer", so PPO is available as an experimental trainer.

• https://raw.githubusercontent.com/huggingface/trl/main/README.md recorded 2026-09-28

  The README states each trainer "natively supports distributed training methods like DDP, DeepSpeed ZeRO, and FSDP" and that TRL "Leverages [🤗 Accelerate](https://github.com/huggingface/accelerate) to scale from single GPU to multi-node clusters", with PEFT LoRA/QLoRA and SFTTrainer, GRPOTrainer, DPOTrainer, KTOTrainer and RewardTrainer examples. Searched the body for FP8 and BF16: none.