AI Potluck
Back to Gap Map Infrastructure / ML orchestration

OpenPAI

Microsoft
open source / Overall score: 2.8

Cluster management and resource scheduling platform for AI, originally from Microsoft Research. Users submit deep-learning jobs to a shared GPU cluster; OpenPAI schedules them with HiveD, a scheduler extender that models each tenant as a virtual cluster and guarantees not just a GPU count but the topology those GPUs sit in, so an eight-GPU job that must run inside one node gets a placement that satisfies it.

The HiveD scheduler is vendored at src/hivedscheduler and also published separately as microsoft/hivedscheduler.

Openness

5 high confidence
5.0
license
MIT(OSI
source
public(microsoft/pai carries the platform and vendors its scheduler at src/hivedscheduler)
core features withheld
no — a 2,498-entry recursive tree carries no path matching enterprise, ee, commercial or proprietary

The OpenPAI LICENSE is the standard MIT grant under a Microsoft Corporation copyright line, read from the body rather than the GitHub label. A full scan of the repository tree finds no enterprise, ee or commercial path, and Microsoft sells no paid edition, so nothing is withheld from what is published.

  • https://api.github.com/repos/microsoft/pai/git/trees/master?recursive=1 recorded 2026-09-15

    Full untruncated recursive tree of the default branch, 2,498 entries. Licenses are the root LICENSE and vendored dependencies. No path matches enterprise, ee, commercial or proprietary. A tree lists paths; it is cited for that and for nothing about the vendor's offerings.

  • https://github.com/microsoft/pai recorded 2026-09-15

    The microsoft/pai repository page. Microsoft publishes no commercial OpenPAI edition, and the repository is the whole platform; this is cited for that and for the license.

  • https://raw.githubusercontent.com/microsoft/pai/master/LICENSE recorded 2026-09-17

    The LICENSE body, read rather than the classifier label: the verbatim MIT text under "Copyright (c) Microsoft Corporation. All rights reserved.", indented four spaces throughout.

Adoption

2 low confidence
2.0

OpenPAI ships as container images and a Helm chart rather than through a package registry, leaving GitHub stars as the only adoption signal. A star is not a use, so the signal shows attention rather than real usage.

Capability

4 medium confidence
4.0

HiveD decides which GPUs on which machines a job receives and guarantees the topology of that set, not just its size, so an eight-GPU job that must run inside one node is placed only where that holds. Its manual describes cell types representing topology above the node level, but that is a description rather than evidence that one job is ever granted a reserved multi-machine set.

  • https://github.com/microsoft/pai recorded 2026-09-15

    Repository page for microsoft/pai, establishing that the platform being scored is the one HiveD schedules for.

  • https://raw.githubusercontent.com/microsoft/hivedscheduler/master/doc/user-manual.md recorded 2026-09-15

    HiveD's user manual: a cellType 'defines a resource topology of a skuType', with skuTypes as leaf cellTypes that 'do not have internal topology anymore' and larger cellTypes composed above them. Establishes that the guaranteed unit is a topology cell rather than a GPU count, and that cells exist above the single-node level.

  • https://raw.githubusercontent.com/microsoft/hivedscheduler/master/README.md recorded 2026-09-15

    HiveD's own README: 'HiveD is a scheduler for deep learning workloads', a Kubernetes scheduler extender for multi-tenant GPU clusters, whose 'killer feature' is topology-aware resource guarantee - the example given is an eight-GPU job that must run within a single node and cannot be allocated by a scheduler that guarantees only a count.

Verified 2026-09-15