Back to Gap Map Model components / Fine-tuning code

Colossal-AI

HPC-AI Tech
open source / Overall score: 3.0

Colossal-AI is a PyTorch library for large-scale distributed training and fine-tuning. It combines data, pipeline, sequence and several forms of tensor parallelism with the ZeRO optimizer and heterogeneous memory management, and ships applications on top of them, among them ColossalChat, an RLHF pipeline, and fine-tuning recipes for large open models. HPC-AI Tech builds it and runs a GPU cloud with Colossal-AI preinstalled.

Openness

5 high confidence
5.0
license
Apache-2.0 (OSI, root LICENSE, with the MIT and BSD notices of bundled third-party code)
source
public (hpcaitech/ColossalAI)
core features withheld
no — HPC-AI Tech sells GPU rental on HPC-AI Cloud with Colossal-AI preinstalled, a separate service running the open library

Colossal-AI is Apache-2.0 and the full library is public. HPC-AI Tech rents GPUs on its own cloud with Colossal-AI preinstalled, but that is a separate service running the same open code, and the README names no paid edition of the library.

  • https://pypi.org/pypi/colossalai/json recorded 2026-11-12

    PyPI lists colossalai 0.5.0, uploaded 2025-06-04, with the license field "Apache Software License 2.0".

  • https://raw.githubusercontent.com/hpcaitech/ColossalAI/main/LICENSE recorded 2026-11-12

    The LICENSE opens "Copyright 2021- HPC-AI Technology Inc." followed by the Apache License 2.0 text, then reproduces the licenses of incorporated code, among them "LICENSE FOR Microsoft Deepspeed" (MIT), "LICENSE FOR NVIDIA Megatron-LM", "LICENSE FOR Flash Attention" (BSD 3-Clause) and Apache-2.0 notices for vLLM and LightLLM.

  • https://raw.githubusercontent.com/hpcaitech/ColossalAI/main/README.md recorded 2026-11-12

    The README's "Instantly Run Colossal-AI on Enterprise-Grade GPUs" section offers "a powerful, pre-configured Colossal-AI environment on HPC-AI Cloud" with B200 and H200 rental priced per hour; the library itself is documented and installed from the repository and PyPI. Searched the body for license key, premium and paid edition: none.

Adoption

2 medium confidence
2.0

Adoption is the trailing month of PyPI downloads of colossalai. Installs from source and the preinstalled images on HPC-AI Cloud are not counted.

Capability

4 medium confidence
4.0

Colossal-AI composes data, pipeline, tensor and sequence parallelism with ZeRO and offloading, and carries RLHF and large-model fine-tuning recipes on top. It sits a step below Megatron-LM, whose published scaling runs reach thousands of GPUs where Colossal-AI's reach tens.

  • https://raw.githubusercontent.com/hpcaitech/ColossalAI/main/README.md recorded 2026-11-12

    The Features list gives "Data Parallelism", "Pipeline Parallelism", "1D, 2D, 2.5D, 3D Tensor Parallelism", "Sequence Parallelism", "Zero Redundancy Optimizer (ZeRO)", "Auto-Parallelism" and "PatrickStar" memory management; the benchmark table runs 7B models on 8 and 70B models on 16 H200 or B200 GPUs; the news lists a "DeepSeek 671B Fine-Tuning Guide" (2025/02) and the applications include "ColossalChat: An Open-Source Solution for Cloning ChatGPT With a Complete RLHF Pipeline".

Verified 2026-11-12, except some scores, still awaiting re-confirmation