LMDeploy
Shanghai AI LaboratoryLMDeploy is a toolkit for compressing, deploying, and serving large language models, built by the InternLM team at Shanghai AI Laboratory. It provides two inference engines: TurboMind, which pairs persistent batching with a blocked KV cache and custom CUDA kernels, and a pure-Python PyTorch engine. Both support weight-only and KV-cache quantization, across families including Llama, Qwen, DeepSeek, and vision-language models such as InternVL.
Openness
5 high confidence- license
- Apache-2.0(OSI)
- source
- public(InternLM/lmdeploy)
- quantization
- weight-only+kv(AWQ/W4A16)
- core features withheld
- no
The repository is public and active, and the LICENSE body is the unmodified Apache-2.0 text. Its README and repo page name no enterprise or paid tier for LMDeploy itself, so nothing is withheld from the published source.
- https://github.com/InternLM/lmdeploy recorded 2026-08-09
repo metadata "license":{"spdxId":"Apache-2.0","name":"Apache License 2.0"}; README documents AWQ, W4A16, tensor parallel, TurboMind and PyTorch engines; no enterprise/paid-tier content in the project's own text (only GitHub's site-wide nav uses those words)
- https://raw.githubusercontent.com/InternLM/lmdeploy/main/LICENSE recorded 2026-09-29
The LICENSE file is the Apache License, "Apache License" "Version 2.0, January 2004" with "TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION".
- https://raw.githubusercontent.com/InternLM/lmdeploy/main/README.md recorded 2026-09-29
The README states "This project is released under the [Apache 2.0 license](LICENSE)." and describes two engines: "LMDeploy has developed two inference engines - [TurboMind]"; searched for enterprise, paid, license key and non-commercial: none.
Adoption
2 medium confidencePyPI downloads dropped sharply and have stayed at that lower level for three months running, so the drop reads as a structural shift rather than a one-month blip. PyPI is the package's only distribution channel, since releases from v0.13.0 onward ship no wheel assets anywhere else, so nothing else absorbs the difference.
- https://api.github.com/repos/InternLM/lmdeploy/releases recorded 2026-08-09
releases v0.13.0, v0.14.0 and v0.15.0 each carry an empty "assets": [] list, so no wheel is distributed outside PyPI
- https://github.com/InternLM/lmdeploy/network/dependents recorded 2026-08-09
356 Repositories (dependent_type=REPOSITORY tab count)
- https://pypistats.org/api/packages/lmdeploy/overall?mirrors=false recorded 2026-08-09
daily download history; summed by month it gives 171,750 for 2026-05, 57,776 for 2026-06 and 54,993 for 2026-07
- https://pypistats.org/api/packages/lmdeploy/recent recorded 2026-08-09
{"data":{"last_day":841,"last_month":50402,"last_week":16260}
- https://pypistats.org/packages/lmdeploy recorded 2026-08-09
Downloads last month: 50,402 / Downloads last week: 16,260 / Downloads last day: 841
Capability
4 medium confidenceLMDeploy's TurboMind and PyTorch engines put it one tier below vLLM, with a comparable production feature set: continuous batching, quantization and broad model coverage. Its own claim of higher throughput than vLLM is a vendor self-benchmark, so the placement instead reflects vLLM's independent MLPerf submission and wider architecture coverage.
- https://github.com/InternLM/lmdeploy recorded 2026-08-09
TurboMind and PyTorch engine descriptions; persistent batching, blocked KV cache, AWQ/W4A16 weight-only and kv quantization, tensor parallel, InternVL/VLM support, 4-bit ~2.4x throughput claim
- https://raw.githubusercontent.com/InternLM/lmdeploy/main/README.md recorded 2026-09-29
README feature list: "persistent batch(a.k.a. continuous batching), blocked KV cache, dynamic split&fuse, tensor parallelism" and "LMDeploy supports weight-only and k/v quantization, and the 4-bit inference performance is 2.4x higher than FP16"; it lists "InternVL3.5 (1B-241BA28B)" among supported models.
Verified 2026-09-24