AI Potluck
Contribute
Model components / Inference code

NVIDIA Dynamo

NVIDIA

Dynamo is NVIDIA's datacenter-scale inference serving stack, the orchestration layer that turns clusters running vLLM, SGLang or TensorRT-LLM into one coordinated system. It disaggregates prefill and decode into independently scalable pools, routes requests by KV cache overlap, offloads KV blocks across GPU, CPU, SSD and remote storage, and autoscales against latency SLAs. It is built in Rust and Python and deploys on Kubernetes.

GitHub reports NOASSERTION because the LICENSE file opens with a NOTICE about DeepSeek test-data files before the full Apache-2.0 text; the body was read in full and the codebase license is Apache-2.0. Verified 2026-09-01 via GitHub, the LICENSE body, the repository README, the PyPI project JSON and pypistats.

Openness

5 high confidence
5.0
license
Apache-2.0(OSI
source
public
core-gated
ungated(the published repo is the whole stack

The LICENSE body opens with a NOTICE that ./lib/llm/tests/data/deepseek-v3.2 files are MIT-licensed DeepSeek derivatives, states "The rest of this codebase is licensed under the Apache License 2.0", and then carries the full unmodified Apache-2.0 text. That NOTICE is why the GitHub API reports NOASSERTION; it adds no term. The repository is public, active and is the product; no component is behind a license key, and the README names no enterprise edition of Dynamo itself.

  • https://raw.githubusercontent.com/ai-dynamo/dynamo/main/LICENSE recorded 2026-09-01

    LICENSE body, read in full: a NOTICE that test-data files under ./lib/llm/tests/data/deepseek-v3.2 are MIT-licensed derivatives of the DeepSeek-V3.2 repository, "The rest of this codebase is licensed under the Apache License 2.0", followed by the complete stock Apache-2.0 text with END OF TERMS AND CONDITIONS at the standard place and nothing appended after the appendix.

  • https://api.github.com/repos/ai-dynamo/dynamo recorded 2026-09-01

    Repo metadata for ai-dynamo/dynamo - archived false, license NOASSERTION (the API could not classify the NOTICE-prefixed file), 7,938 stars, homepage https://docs.nvidia.com/dynamo/latest, pushed 2026-09-01.

  • https://raw.githubusercontent.com/ai-dynamo/dynamo/main/README.md recorded 2026-09-01

    README, SPDX header "Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES ... SPDX-License-Identifier: Apache-2.0"; "The open-source, datacenter-scale inference stack"; read for a paid tier or withheld component of Dynamo itself - none named; Apache-2.0 badge; PyPI ai-dynamo badge; NGC container catalog linked as packaging.

Adoption

2 medium confidence
2.0

PyPI ai-dynamo drew 58,749 downloads in the trailing 30 days, banding at level 2 (10K-100K). The package backlinks to ai-dynamo/dynamo. Dynamo also ships as NGC container images, a channel that publishes no count here, so the band is a floor read on the countable channel - the semantic-kernel shape, with banded_quantity naming what the figure counts.

Capability

4 medium confidence
4.0

One band below the vllm anchor. Its serving-path surface at cluster scale (7x throughput per GPU on GB200 NVL72 reported via SemiAnalysis InferenceX, 2x TTFT via Baseten) is the widest in the category above single-engine scope, but the engine underneath is somebody else's product and Dynamo is inert without one, which is what keeps it off the anchor's band.

  • https://raw.githubusercontent.com/ai-dynamo/dynamo/main/README.md recorded 2026-09-01

    Core capabilities table: Disaggregated Prefill/Decode, KV-Aware Routing, KV Block Manager, ModelExpress weight streaming, SLA Planner, Grove gang scheduling, fault tolerance; "it doesn't replace SGLang, TensorRT-LLM, or vLLM, it turns them into a coordinated multi-node inference system"; key results 7x throughput/GPU (GB200 NVL72), 2x TTFT (Baseten).

Verified 2026-09-01