KVCache.AI

license

Apache-2.0(OSI)

source

public

core features withheld

no — research project

The LICENSE body is the byte-stock Apache-2.0 text (digest identical to the canonical text vLLM ships). The repository is public and very active, and the README describes a research project with no commercial edition, hosted tier or withheld component.

• https://api.github.com/repos/kvcache-ai/ktransformers recorded 2026-09-01

  Repo metadata for kvcache-ai/ktransformers - archived false, license Apache-2.0, 19,430 stars, pushed 2026-09-01.

• https://raw.githubusercontent.com/kvcache-ai/ktransformers/main/LICENSE recorded 2026-10-07

  LICENSE body, read in full: stock Apache License Version 2.0 text, unmodified (digest matches the canonical Apache-2.0 text).

• https://raw.githubusercontent.com/kvcache-ai/ktransformers/main/README.md recorded 2026-10-07

  README, read for a paid tier or gated component (none named): "a research project focused on efficient inference and fine-tuning of large language models through CPU-GPU heterogeneous computing"; kt-kernel Inference and SFT entry points; DeepSeek-R1 671B on single 24GB GPU / 382G DRAM; Ascend NPU, Intel Arc, ROCm, AMX support; SGLang integration roadmap.

GitHub stars are the adoption signal for KTransformers, since the product is installed mainly through hardware-specific source builds. Its PyPI package is confirmed as the project's own, but its downloads are a minority channel next to source installs, so it is left undeclared and stars carry the reading.

• https://api.github.com/repos/kvcache-ai/ktransformers recorded 2026-09-01

  Repo metadata - stargazers_count = 19,430 - for kvcache-ai/ktransformers.

• https://pypistats.org/api/packages/ktransformers/recent recorded 2026-09-01

  "last_month":2223 - the PyPI channel figure the note above declines to use.

KTransformers sits one tier below vLLM, level with LMDeploy and mistral.rs. It can do something vLLM cannot, serve 671B-class MoE models on a single consumer GPU by scheduling experts onto the CPU, and spans four accelerator vendors, but it is not built for datacenter throughput and increasingly delivers its serving path through an SGLang integration rather than standing alone.

• https://raw.githubusercontent.com/kvcache-ai/ktransformers/main/README.md recorded 2026-10-07

  Updates timeline and capabilities: DeepSeek-R1/V3 on single 24GB GPU + 382G DRAM at 3~28x speedup; AMX-Int8/BF16 kernels; FP8 GPU kernel; 3-layer prefix cache; multi-concurrency; ROCm, Intel Arc, Ascend NPU support; SGLang integration; GLM-5.3, MiniMax-M3, Kimi-K2.5 day-0 support.