AI Potluck
Model components / Inference code

Text Generation Inference

Hugging Face

Text Generation Inference (TGI) is Hugging Face's LLM serving toolkit, a Rust, Python, and gRPC server purpose-built for the HF model ecosystem. It supports continuous batching, tensor parallelism, Flash Attention, token streaming, and quantization (GPTQ, AWQ, EETQ), and has powered Hugging Face's Inference Endpoints and Hugging Chat. Its final release was v3.3.7 (2025-12-19) and the repository was archived 2026-03-21, with Hugging Face pointing new deployments to vLLM, SGLang, llama.cpp, and MLX.

The project is archived, so this entry is verified against the final release rather than an active repository. Verified 2026-08-09 via GitHub, the LICENSE body, and the releases page.

Openness

5 high confidence
5.0
license
Apache-2.0(OSI)
source
public
core-gated
ungated(Apache-2.0 across the repo, no ee/ or enterprise directory and no license key
note
was briefly under restrictive HFOIL license in 2023, reverted to Apache-2.0

The current LICENSE body is standard Apache-2.0, not the restrictive HFOIL license the project briefly carried in 2023. The repository page shows a "Public archive" label, so the source remains public despite the archive. Nothing is withheld from that source: there is no ee/ or enterprise directory, no closed package and no license key, and Hugging Face's hosted Inference Endpoints run this code rather than hold features back from it. That the project is archived and in maintenance mode is a fact about its upkeep rather than its openness, and does not change the score.

  • https://github.com/huggingface/text-generation-inference/blob/main/LICENSE recorded 2026-08-11

    Apache License Version 2.0, January 2004; Copyright 2022 Hugging Face

  • https://github.com/huggingface/text-generation-inference recorded 2026-08-11

    "This repository was archived by the owner on Mar 21, 2026. It is now read-only." with a "Public archive" label, confirming the source remains publicly readable. Licensed Apache-2.0. Top-level directories are .github, assets, backends, benchmark, clients/python, docs, integration-tests, launcher, load_tests, nix, proto, router, server -- no ee/ or enterprise directory and no license-key gate. The README notes text-generation-inference is in maintenance mode and points users at vLLM and SGLang. Hugging Face's own products (HuggingChat, Inference API, Inference Endpoints) are named as places TGI runs in production, not as tiers withholding features from this source.

Adoption

3 medium confidence
3.0

README states production use at Hugging Face and explicitly recommends vLLM/SGLang/llama.cpp/MLX going forward, confirming a real but sunsetting footprint.

  • https://github.com/huggingface/text-generation-inference recorded 2026-08-09

    'Used in production at Hugging Face to power Hugging Chat, the Inference API and Inference Endpoints.' and 'This approach is now adopted by downstream inference engines, which we contribute to and recommend using going forward: vllm, SGLang, as well as local engines with inter-compatibility such as llama.cpp or MLX.'

Capability

3 medium confidence
3.0

Development has stopped: the repository is archived and the feature set is frozen at continuous batching, tensor parallelism, Flash Attention, token streaming, GPTQ/AWQ/EETQ quantization, OpenTelemetry tracing and Prometheus metrics, while vLLM and SGLang keep advancing. That holds it mid-pack rather than at the frontier.

Verified 2026-08-09