TensorRT-LLM

license

Apache-2.0(OSI, stated in LICENSE

source

public

core-gated

ungated

caveat

runs only on NVIDIA GPUs (vendor-hardware lock, not a license restriction)

Apache-2.0 across the repository. The LICENSE body opens "This project is licensed under the Apache 2.0 license" and then appends the terms of bundled third-party code, MIT, BSD-3-Clause and an LTX-2 Community License scoped to a single visual-generation model directory, which is why GitHub's classifier reports NOASSERTION. The published tree carries the C++ runtime and kernels in cpp/ rather than a stub, so nothing is held back from the source; NVIDIA sells NIM and enterprise support as separate products.

• https://github.com/NVIDIA/TensorRT-LLM recorded 2026-08-09

  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.

• https://github.com/NVIDIA/TensorRT-LLM/blob/main/LICENSE recorded 2026-08-09

  This project is licensed under the Apache 2.0 license, whose full license text is available below.

• https://github.com/NVIDIA/TensorRT-LLM recorded 2026-08-09

  {"name":"cpp","path":"cpp","contentType":"directory"} in the repository file tree, alongside 3rdparty/, docker/ and examples/ - the C++ runtime and kernels are in-tree, with no stub or binary-only component.

14,342 GitHub stars, and the comparison write-ups consistently frame TensorRT-LLM as the specialist choice: the highest raw throughput on standardized NVIDIA fleets, but narrower reach than vLLM or SGLang because of its compilation overhead and single-vendor lock-in. No clean standalone user or download count exists to band directly, so level 3 reflects significant but specialized production usage.

• https://github.com/NVIDIA/TensorRT-LLM recorded 2026-08-09

  "stargazerCount":14342

• https://www.yottalabs.ai/post/best-llm-inference-engines-in-2026-vllm-tensorrt-llm-tgi-and-sglang-compared recorded 2026-08-09

  Because TensorRT-LLM is built specifically for NVIDIA hardware, it can achieve extremely high performance in environments where infrastructure is standardized around NVIDIA GPU clusters. However, this hardware specialization can also make it less flexible for teams running heterogeneous infrastructure across multiple cloud environments.

• https://theaiengineer.substack.com/p/vllm-vs-ollama-vs-sglang-vs-tensorrt recorded 2026-08-09

  TensorRT-LLM squeezes maximum performance from NVIDIA hardware but demands 1-2 weeks of setup time and locks you into a single vendor.

NVIDIA's own MLPerf Inference v4.0 recap states that the HGX H100 platform "delivered nearly 3x more performance on the GPT-J test compared to the prior round thanks to our TensorRT-LLM software." An independent H100 bake-off on Llama 3.3 70B at FP8 puts TensorRT-LLM ahead on raw throughput at 2,100 tokens per second, against vLLM's 1,850 and SGLang's 1,920, at the cost of a much longer cold start, roughly 28 minutes against about 1 minute. The same source cites a newer engine, TokenSpeed, claiming a 9-11% edge over TensorRT-LLM on B200 for agentic workloads.

• https://www.hpcwire.com/2024/03/28/mlperf-inference-4-0-results-showcase-genai-nvidia-still-dominates/ recorded 2026-08-09

  delivered nearly 3x more performance on the GPT-J test compared to the prior round thanks to our TensorRT-LLM software

• https://www.spheron.network/blog/vllm-vs-tensorrt-llm-vs-sglang-benchmarks/ recorded 2026-08-09

  Use TensorRT-LLM if you have a single model in long-term production and throughput is paramount.