TorchTitan
PyTorch Foundation (Linux Foundation)TorchTitan is a PyTorch-native platform for pretraining and post-training generative models at scale. One readable codebase takes a model from pretraining through supervised fine-tuning to reinforcement learning, composing FSDP2, tensor, pipeline, context and expert parallelism through DeviceMesh and DTensor, with float8 and FP4 training and recipes for open model families. The PyTorch team develops it in the pytorch organization.
Openness
5 high confidence- license
- BSD-3-Clause (OSI)
- source
- public (pytorch/torchtitan)
- core features withheld
- no — developed in the PyTorch project's own repository, with no company edition or hosted tier
TorchTitan is BSD-3-Clause and developed in the public pytorch repository, alongside PyTorch itself. There is no paid edition or hosted tier.
- https://raw.githubusercontent.com/pytorch/torchtitan/main/LICENSE recorded 2026-11-12
The LICENSE is the standard "BSD 3-Clause License" text, "(c) Meta Platforms, Inc. and affiliates.", with no added field-of-use or commercial restriction.
- https://raw.githubusercontent.com/pytorch/torchtitan/main/pyproject.toml recorded 2026-11-12
The manifest builds the "torchtitan" package with license = {file = "LICENSE"} and the author "PyTorch Team", packages@pytorch.org.
- https://raw.githubusercontent.com/pytorch/torchtitan/main/README.md recorded 2026-11-12
The README describes "a clean-room, PyTorch-native implementation of large-scale training" installed by cloning the repository, with a BSD 3-Clause license badge. Searched the body for enterprise, pricing, paid, hosted and license key: none.
Adoption
2 medium confidenceAdoption is the trailing month of PyPI downloads of torchtitan. The quick start installs from a git checkout against PyTorch nightly builds, which PyPI does not see, so those installs are not counted.
- https://pypistats.org/api/packages/torchtitan/recent recorded 2026-11-12
last_month 73,750, last_week 24,589, last_day 2,526 for the torchtitan package.
Capability
4 medium confidenceTorchTitan composes five kinds of parallelism with low-precision training and carries a model from pretraining through fine-tuning to RL, with recipes up to 671B parameters. It sits a step below Megatron-LM, whose published scaling runs reach thousands of GPUs while TorchTitan's paper reports 512.
- https://arxiv.org/abs/2410.06511 recorded 2026-11-12
The ICLR 2025 paper abstract reports "accelerations of 65.08% with 1D parallelism at the 128-GPU scale (Llama 3.1 8B), an additional 12.59% with 2D parallelism at the 256-GPU scale (Llama 3.1 70B), and an additional 30% with 3D parallelism at the 512-GPU scale (Llama 3.1 405B) on NVIDIA H100 GPUs".
- https://raw.githubusercontent.com/pytorch/torchtitan/main/README.md recorded 2026-11-12
The README lists "FSDP2, HSDP, and DDP", "Tensor parallel, including async TP", "Pipeline parallel: 1F1B, Interleaved 1F1B, GPipe", "Context parallel", "Expert parallel", MXFP8 and NVFP4 training, TitanRL running "the same model definitions and kernels in the trainer and in vLLM generation", and supported models including Llama 3 "1B – 405B" and DeepSeek V3 "16B, 236B, 671B".
Verified 2026-11-12, except some scores, still awaiting re-confirmation