llm-d

license

Apache-2.0(OSI)

source

public

core-gated

ungated(CNCF project

The LICENSE body is the byte-stock Apache-2.0 text. The repository is public and active, the guides and Helm charts are in-tree, and no component sits behind a license key or enterprise edition; the vendors' paid platforms that bundle llm-d are separate products, the langchain shape.

• https://raw.githubusercontent.com/llm-d/llm-d/main/LICENSE recorded 2026-09-01

  LICENSE body, read in full: stock Apache License Version 2.0 text, unmodified (digest matches the canonical Apache-2.0 text).

• https://api.github.com/repos/llm-d/llm-d recorded 2026-09-01

  Repo metadata for llm-d/llm-d - archived false, license Apache-2.0, 4,375 stars, homepage https://www.llm-d.ai, pushed 2026-09-01.

• https://raw.githubusercontent.com/llm-d/llm-d/main/README.md recorded 2026-09-01

  README, read for a paid tier or withheld component (none named): "a high-performance distributed inference serving stack optimized for production deployments on Kubernetes"; CNCF Sandbox, founded by Red Hat, Google Cloud, IBM Research, CoreWeave, NVIDIA; intelligent routing, tiered KV cache, prefill/decode disaggregation, wide expert-parallelism, SLO-aware autoscaling, batch APIs.

4,375 GitHub stars, in the 1K-10K band of the stars scale. The product ships through container images and Helm charts, channels that publish no count the pipeline can read, so stars are the last honest route; production deployments named in the README (Tesla/Red Hat, AWS, Oracle, Google benchmarks) indicate standing the star count understates, and raising the level on them would need its own evidenced reported_traction judgment, which this packet does not make.

• https://api.github.com/repos/llm-d/llm-d recorded 2026-09-01

  Repo metadata - stargazers_count = 4,375 - for llm-d/llm-d.

One band below the vllm anchor, level with dynamo, which is the same architectural shape: a serving stack whose forward pass belongs to somebody else's engine. Its cross-accelerator production benchmarks are the widest validation surface of the orchestration entrants, but without an engine of its own it does not reach the anchor's band.

• https://raw.githubusercontent.com/llm-d/llm-d/main/README.md recorded 2026-09-01

  Performance highlights: 3x throughput / 2x TTFT with prefix-cache-aware routing (Llama 3.1 70B, 4x MI300X, Tesla/Red Hat); 70% higher tokens/sec with P/D disaggregation vs standard vLLM (AWS B200); 50k tok/s wide-EP on 16x16 B200; 13.9x with hierarchical KV offload; four core themes - routing, KV management, serving large models, operational excellence.