FlashInfer

license

Apache-2.0(OSI)

source

public

core features withheld

yes — the default flashinfer-python install requires NVIDIA's proprietary nvidia-cutlass-dsl and compiles or downloads kernels on first use

closed-dependency

nvidia-cutlass-dsl(NVIDIA proprietary license, a base dependency of flashinfer-python)

FlashInfer's own code is Apache-2.0 and public, but the default install is not open end to end. flashinfer-python lists NVIDIA's nvidia-cutlass-dsl, a proprietary package, among its base dependencies, and the core package compiles or downloads kernels on first use, with the TensorRT-LLM and cuDNN paths among its backends. A closed component that the default build requires makes this open core rather than open source.

• https://api.github.com/repos/flashinfer-ai/flashinfer recorded 2026-08-18

  Repo metadata - license spdx_id Apache-2.0, private false, archived false, default branch main - for flashinfer-ai/flashinfer.

• https://pypi.org/pypi/flashinfer-python/json recorded 2026-10-08

  PyPI metadata for flashinfer-python lists "nvidia-cutlass-dsl>=4.6.2a0" in requires_dist without an extra marker, beside its "Apache-2.0" license expression.

• https://pypi.org/pypi/nvidia-cutlass-dsl/json recorded 2026-10-08

  PyPI metadata for nvidia-cutlass-dsl carries the classifier "License :: Other/Proprietary License".

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/LICENSE recorded 2026-10-08

  The LICENSE file is the Apache License text, "Apache License" and "Version 2.0, January 2004".

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/pyproject.toml recorded 2026-10-08

  pyproject.toml declares license = "Apache-2.0" and takes its base dependencies from requirements.txt, noting "BASE dependencies (requirements.txt)".

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/README.md recorded 2026-10-08

  README quickstart is "pip install flashinfer-python", described as the "Core package that compiles/downloads kernels on first use", with backends "including FlashAttention-2/3, cuDNN, CUTLASS, and TensorRT-LLM".

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/requirements.txt recorded 2026-10-08

  requirements.txt, the base dependency list, includes "nvidia-cutlass-dsl>=4.6.2a0" with no extra marker.

Adoption is measured from PyPI downloads of the `flashinfer-python` package.

• https://pypistats.org/api/packages/flashinfer-python/recent recorded 2026-08-18

  last_month downloads = 4,830,965 for flashinfer-python

FlashInfer provides JIT-specialized attention, GEMM and MoE kernels across four backend implementations, and several serving engines - SGLang, vLLM, MLC-LLM and ScaleLLM - declare it as a dependency. It covers several operator families rather than one, but only on NVIDIA GPUs from Turing through Blackwell, which keeps it one step below Apache TVM's broader hardware reach.

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/README.md recorded 2026-09-30

  README describes FlashInfer as "a library and kernel generator for inference that delivers state-of-the-art performance across diverse GPU architectures" with "unified APIs for attention, GEMM, and MoE operations" over "FlashAttention-2/3, cuDNN, CUTLASS, and TensorRT-LLM" backends, and lists "Support for SM75 (Turing) and later (through Blackwell)".

• https://raw.githubusercontent.com/flashinfer-ai/flashinfer/main/README.md recorded 2026-09-30

  README lists under Adoption "FlashInfer powers inference in:" SGLang, vLLM, TensorRT-LLM, TGI, MLC-LLM, LightLLM, lorax and ScaleLLM, and states "FlashInfer is inspired by [FlashAttention]".

• https://raw.githubusercontent.com/vllm-project/vllm/main/requirements/cuda.txt recorded 2026-09-30

  vLLM CUDA requirements pin "flashinfer-python==0.7.0" and "flashinfer-cubin==0.7.0" with "--extra-index-url https://flashinfer.ai/whl/", so a consumer states the dependency from its own build requirements.