LMDeploy
Shanghai AI LaboratoryLMDeploy is a toolkit for compressing, deploying, and serving large language models, built by the InternLM team at Shanghai AI Laboratory. It provides two inference engines: TurboMind, which pairs persistent batching with a blocked KV cache and custom CUDA kernels, and a pure-Python PyTorch engine. Both support weight-only and KV-cache quantization, across families including Llama, Qwen, DeepSeek, and vision-language models such as InternVL.
Verified 2026-08-09 via GitHub, the LICENSE body, and PyPI download statistics.
Openness
5 high confidence- license
- Apache-2.0(OSI)
- source
- public(InternLM/lmdeploy)
- quantization
- weight-only+kv(AWQ/W4A16)
- core-gated
- ungated
The repository is public and active, the LICENSE body is the unmodified Apache-2.0 text, and GitHub's own repository metadata reports the same Apache-2.0 identifier. No pricing or enterprise page exists for the project and no paid tier is mentioned anywhere: the only "enterprise" and "pricing" strings on the repo page belong to GitHub's site navigation rather than to LMDeploy, so nothing is withheld from the published source. AWQ/W4A16 weight-only and KV-cache quantization are documented on the repo page.
- https://github.com/InternLM/lmdeploy recorded 2026-08-09
repo metadata "license":{"spdxId":"Apache-2.0","name":"Apache License 2.0"}; README documents AWQ, W4A16, tensor parallel, TurboMind and PyTorch engines; no enterprise/paid-tier content in the project's own text (only GitHub's site-wide nav uses those words)
- https://raw.githubusercontent.com/InternLM/lmdeploy/main/LICENSE recorded 2026-08-09
Apache License Version 2.0, January 2004, http://www.apache.org/licenses/
Adoption
2 medium confidencePyPI downloads stepped down sharply and have since held at the new level for three months: 171,750 in the month before the step, then 57,776, then 54,993, and 50,402 over the trailing 30 days. That sustained volume, rather than any single reading, is what places it at level 2, the 10K-100K band. No second distribution channel absorbs the difference: releases from v0.13.0 onward attach no wheel assets, so PyPI is the package's only route.
- https://pypistats.org/packages/lmdeploy recorded 2026-08-09
Downloads last month: 50,402 / Downloads last week: 16,260 / Downloads last day: 841
- https://pypistats.org/api/packages/lmdeploy/recent recorded 2026-08-09
{"data":{"last_day":841,"last_month":50402,"last_week":16260}
- https://github.com/InternLM/lmdeploy/network/dependents recorded 2026-08-09
356 Repositories (dependent_type=REPOSITORY tab count)
- https://pypistats.org/api/packages/lmdeploy/overall?mirrors=false recorded 2026-08-09
daily download history; summed by month it gives 171,750 for 2026-05, 57,776 for 2026-06 and 54,993 for 2026-07
- https://api.github.com/repos/InternLM/lmdeploy/releases recorded 2026-08-09
releases v0.13.0, v0.14.0 and v0.15.0 each carry an empty "assets": [] list, so no wheel is distributed outside PyPI
Capability
4 medium confidenceThe feature set is the TurboMind and PyTorch engines, persistent and continuous batching, a blocked KV cache, AWQ/W4A16 weight-only and KV-cache quantization, tensor parallelism, broad LLM and VLM architecture coverage, and a vendor-reported 2.4x speedup for 4-bit inference over FP16. It sits one tier below vLLM, which holds the top capability score in this category. The README's own claim of up to 1.8x higher request throughput than vLLM is a vendor self-benchmark; the placement rests instead on vLLM's independent MLPerf submission and its broader architecture coverage.
- https://github.com/InternLM/lmdeploy recorded 2026-08-09
TurboMind and PyTorch engine descriptions; persistent batching, blocked KV cache, AWQ/W4A16 weight-only and kv quantization, tensor parallel, InternVL/VLM support, 4-bit ~2.4x throughput claim
Verified 2026-08-09