AI Potluck
Back to Gap Map Infrastructure / ML orchestration

OpenPAI

Microsoft
open source / Overall score: 2.8

Cluster management and resource scheduling platform for AI, originally from Microsoft Research. Users submit deep-learning jobs to a shared GPU cluster; OpenPAI schedules them with HiveD, a scheduler extender that models each tenant as a virtual cluster and guarantees not just a GPU count but the topology those GPUs sit in, so an eight-GPU job that must run inside one node gets a placement that satisfies it.

The HiveD scheduler is vendored at src/hivedscheduler and also published separately as microsoft/hivedscheduler. Verified 2026-09-15 via the repository, its recursive tree and the HiveD scheduler README.

Openness

5 high confidence
5.0
license
MIT(OSI
source
public(microsoft/pai carries the platform and vendors its scheduler at src/hivedscheduler)
core-gated
ungated(a 2,498-entry recursive tree carries no path matching enterprise, ee, commercial or proprietary)

MIT, read from the body rather than the label: the text is the standard MIT grant under a Microsoft Corporation copyright line. The 2,498-entry recursive tree carries no enterprise, ee or commercial path, and Microsoft sells no paid OpenPAI, so nothing is withheld from what is published.

  • https://github.com/microsoft/pai recorded 2026-09-15

    The microsoft/pai repository page. Microsoft publishes no commercial OpenPAI edition, and the repository is the whole platform; this is cited for that and for the license.

  • https://api.github.com/repos/microsoft/pai/git/trees/master?recursive=1 recorded 2026-09-15

    Full untruncated recursive tree of the default branch, 2,498 entries. Licenses are the root LICENSE and vendored dependencies. No path matches enterprise, ee, commercial or proprietary. A tree lists paths; it is cited for that and for nothing about the vendor's offerings.

Adoption

2 low confidence
2.0

2,690 GitHub stars. The project ships as container images and a Helm chart and publishes no package-registry artifact, so no usage-volume channel exists and the band rests on stars, which measure attention rather than use. The stars route caps at level 3 for that reason.

Capability

4 medium confidence
4.0

HiveD decides which GPUs on which machines a job receives, and guarantees the topology of that set rather than only its size - an eight-GPU job that must run inside one node is placed only where that holds. That is selecting the hardware, which is band 4. It is NOT banded 5: the cited manual establishes that cells represent topology above the node level, and a representation is not the same as evidence that one job receives a multi-machine set reserved together. The band says what the documentation shows and stops there. It anchors this band rather than resting on a peer, now that nvidia-run-ai has moved to 5 on its own scheduler documentation.

  • https://raw.githubusercontent.com/microsoft/hivedscheduler/master/README.md recorded 2026-09-15

    HiveD's own README: 'HiveD is a scheduler for deep learning workloads', a Kubernetes scheduler extender for multi-tenant GPU clusters, whose 'killer feature' is topology-aware resource guarantee - the example given is an eight-GPU job that must run within a single node and cannot be allocated by a scheduler that guarantees only a count.

  • https://github.com/microsoft/pai recorded 2026-09-15

    Repository page for microsoft/pai, establishing that the platform being scored is the one HiveD schedules for.

  • https://raw.githubusercontent.com/microsoft/hivedscheduler/master/doc/user-manual.md recorded 2026-09-15

    HiveD's user manual: a cellType 'defines a resource topology of a skuType', with skuTypes as leaf cellTypes that 'do not have internal topology anymore' and larger cellTypes composed above them. Establishes that the guaranteed unit is a topology cell rather than a GPU count, and that cells exist above the single-node level.

Verified 2026-09-15