vLLM
vLLM (UC Berkeley / vLLM team)High-throughput LLM serving engine built on PagedAttention, which treats KV cache as virtual memory pages to near-eliminate memory waste during batched inference. Supports continuous batching, tensor parallelism, speculative decoding, and 40+ model architectures. The de facto standard for self-hosted production model serving, used by Anyscale, Databricks, and most open source inference deployments.
v0.22.0, released May 29, 2026 (GitHub releases). Confirmed to exist as of June 2026.
Openness
5 high confidence- license
- Apache-2.0(OSI)
- source
- public
- governance
- vLLM-project(community,PyTorch-ecosystem adjacent)
- https://github.com/vllm-project/vllm recorded 2026-06-04
flagship phase-C verification source
- https://www.programming-helper.com/tech/vllm-2026-open-source-llm-inference-engine recorded 2026-06-04
flagship phase-C verification source
Adoption
4 high confidence~3M+ PyPI downloads/month as of 2026; de facto standard open inference engine with enterprise adoption (Amazon, Roblox, Stripe). 81.9k GitHub stars corroborates but is not the primary basis. Level 4 (1-10M) on download volume; could exceed if cumulative installs counted.
- https://pypistats.org/packages/vllm recorded 2026-06-04
flagship phase-C verification source
- https://pypi.org/project/vllm/ recorded 2026-06-04
flagship phase-C verification source
- https://awesomeagents.ai/tools/best-open-source-llm-inference-servers-2026/ recorded 2026-06-04
flagship phase-C verification source
Capability
5 high confidenceReference PagedAttention implementation; used in MLPerf Inference submissions (Red Hat OpenShift AI on GPT-J-6b and llama-2-70b). Broad model/hardware coverage (200+ architectures) per repo. Top-tier among open inference engines.
- https://www.redhat.com/en/blog/accelerating-generative-ai-adoption-red-hat-openshift-ai-achieves-impressive-results-mlperf-inference-benchmarks-vllm-runtime recorded 2026-06-04
flagship phase-C verification source
- https://github.com/vllm-project/vllm recorded 2026-06-04
flagship phase-C verification source
Unchanged since 2026-06-09 (last edited, not re-checked)