Back to Gap Map Model components / Inference code

Xinference

Xorbits
open core / Overall score: 3.5

Xinference is a platform for serving open models on anything from a laptop to a multi-machine cluster. It runs language, embedding, reranking, image, audio and video models on engines including vLLM, SGLang, llama.cpp, Transformers and MLX, behind OpenAI- and Anthropic-compatible APIs, and adds a web interface for deploying and managing them. It comes from the Xorbits project.

Openness

4 high confidence
4.0
license
Apache-2.0 (OSI)
source
public (the community edition builds and runs from the repository)
core features withheld
yes — Xinference Enterprise is the same engine plus commercial extensions, adding multi-cluster, multi-region and high-availability scaling the community edition lacks

Xinference's code is Apache-2.0, and the community edition builds and runs from the public repository. The vendor also sells an enterprise edition it describes as the same engine plus commercial extensions, with multi-cluster and high-availability scaling the open edition does not have, so part of the product is held back.

  • https://raw.githubusercontent.com/xorbitsai/inference/HEAD/LICENSE recorded 2026-11-12

    The LICENSE body is the stock Apache License 2.0 text with no added condition.

  • https://raw.githubusercontent.com/xorbitsai/inference/HEAD/README.md recorded 2026-11-12

    The README calls Xinference "an open-source distributed inference platform", starts it with "pip install xinference", and has a section "Xinference Enterprise" inviting readers to "Deploy Xinference Enterprise in your own cloud or data center", with a link for "Business License Inquiry".

  • https://ungh.cc/repos/xorbitsai/inference/files/main recorded 2026-11-12

    The repository file listing shows the xinference package, web UI and deployment files, and no ee, enterprise, commercial or proprietary path; the enterprise extensions are not in this repository.

  • https://xinference.co/pricing.html recorded 2026-11-12

    The pricing page's "Community vs Enterprise" table describes the Community engine as "Apache-2.0 inference engine" and Enterprise as "The same engine, plus commercial extensions", with scaling "Single cluster" against "Multi-cluster, multi-region, HA" and hardware "NVIDIA only (validated)" against "NVIDIA, AMD, Intel, CPU".

Adoption

3 medium confidence
3.0

Xinference ships mainly as a Docker image. The reading combines Docker Hub pulls with PyPI downloads, and the Docker figure is a cumulative count since the image was first published rather than a monthly one.

Capability

4 medium confidence
4.0

Xinference adds what a single engine lacks: a model catalog, multi-machine deployment, prefill/decode separation and a management interface, across many model types. It hands the model's forward pass to vLLM, SGLang or another engine rather than running it itself, so it sits a step below vLLM.

  • https://raw.githubusercontent.com/xorbitsai/inference/HEAD/README.md recorded 2026-11-12

    The README's feature table lists "300+ built-in models", engines "vLLM, SGLang, TRT-LLM, Transformers, llama.cpp, or MLX", distributed serving with "heterogeneous prefill/decode separation", "OpenAI- and Anthropic-compatible endpoints", shared prefix caches "with Xavier" and replica-by-replica reloads.

Verified 2026-11-12, except some scores, still awaiting re-confirmation