Megatron-LM

license

BSD-3-Clause(OSI, NVIDIA core)+Apache-2.0/MIT(OSI, vendored components)

source

public

restriction

attribution/change-notice only

• https://github.com/NVIDIA/Megatron-LM/blob/main/LICENSE recorded 2026-06-04

  flagship phase-C verification source

• https://github.com/NVIDIA/Megatron-LM recorded 2026-06-04

  flagship phase-C verification source

Foundational large-scale training library; directly inspired/underpins Colossal-AI, HuggingFace Accelerate, NVIDIA NeMo. Used to train models 2B-462B params across thousands of GPUs. User base is concentrated among large-model training teams (smaller absolute headcount than inference tools). 16.6k stars. Level 3 reflects specialist-but-influential reach; no clean download count verified.

• https://github.com/NVIDIA/Megatron-LM recorded 2026-06-04

  flagship phase-C verification source

• https://developer.nvidia.com/megatron-core recorded 2026-06-04

  flagship phase-C verification source

State-of-the-art parallelism (TP/PP/DP/EP/CP), mixed precision FP16/BF16/FP8/FP4; benchmarked training of 462B-param model on 6144 H100 GPUs. Reference-grade large-scale transformer training infra. No MLPerf-Training submission verified for the library itself, so basis is feature_matrix plus NVIDIA-reported scaling benchmarks.

• https://github.com/NVIDIA/Megatron-LM recorded 2026-06-04

  flagship phase-C verification source

• https://developer.nvidia.com/megatron-core recorded 2026-06-04

  flagship phase-C verification source