Triton Inference Server
NVIDIATriton Inference Server serves models from many frameworks, including TensorRT, PyTorch, ONNX, OpenVINO, Python and RAPIDS FIL, over HTTP/REST and gRPC. It runs several models at once with dynamic and sequence batching, chains them into ensembles, and works on NVIDIA GPUs, x86 and ARM CPUs and AWS Inferentia. NVIDIA develops it and publishes release containers on NGC.
Openness
5 medium confidence- license
- BSD-3-Clause (OSI)
- source
- public (the server and its build scripts; backends live in separate repositories)
- core features withheld
- no — NVIDIA sells enterprise support through the separate NVIDIA AI Enterprise suite; no enterprise path in the repository tree
Triton's server code is BSD-3-Clause and builds from the public repository, with backends for open frameworks such as PyTorch, ONNX and OpenVINO alongside NVIDIA's own TensorRT. NVIDIA sells enterprise support for it through NVIDIA AI Enterprise, a separate subscription, rather than a fuller edition of the server.
- https://raw.githubusercontent.com/triton-inference-server/server/HEAD/LICENSE recorded 2026-11-12
The LICENSE body is the BSD 3-Clause text, "Copyright (c) 2018-2026, NVIDIA CORPORATION. All rights reserved."
- https://raw.githubusercontent.com/triton-inference-server/server/HEAD/README.md recorded 2026-11-12
The README carries a BSD3 license badge, calls Triton "open source inference serving software", says the current release is 2.74.0, and states "Need enterprise support? NVIDIA global support is available for Triton Inference Server with the NVIDIA AI Enterprise software suite."
- https://ungh.cc/repos/triton-inference-server/server/files/main recorded 2026-11-12
The repository file listing shows build.py, docs/customization_guide/build.md and the server sources, and no ee, enterprise, commercial or proprietary path.
Adoption
3 low confidenceTriton is deployed mostly from NVIDIA's NGC containers, which publish no pull count, and the tritonclient package on PyPI is a client for talking to a server rather than the server itself. GitHub stars are the only signal left, and a star is not a use.
- https://pypi.org/pypi/tritonclient/json recorded 2026-11-12
The PyPI package tritonclient 2.74.0 is summarized as "Python client library and utilities for communicating with Triton Inference Server".
- https://ungh.cc/repos/triton-inference-server/server recorded 2026-11-12
"stars":11136,"forks":1868, pushed 2026-11-10, for triton-inference-server/server
Capability
4 medium confidenceTriton serves models from almost any framework on GPUs, CPUs and Inferentia, with dynamic batching, model pipelines and in-process APIs for edge use. Its README documents none of the language-model techniques vLLM is built on, such as paged attention and continuous batching, which it reaches only through separate backends, so it sits a step below vLLM.
- https://raw.githubusercontent.com/triton-inference-server/server/HEAD/README.md recorded 2026-11-12
The README lists support for "TensorRT, PyTorch, ONNX, OpenVINO, Python, RAPIDS FIL", "NVIDIA GPUs, x86 and ARM CPU, or AWS Inferentia", "Concurrent model execution", "Dynamic batching", "Sequence batching", ensembles and Business Logic Scripting, "HTTP/REST and GRPC inference protocols", C and Java APIs, and metrics.
Verified 2026-11-12