Model components / Inference code

Text Generation Inference

Hugging Face's production LLM serving solution, purpose-built for the HF model ecosystem. Supports continuous batching, tensor parallelism, Flash Attention, and quantization (GPTQ, AWQ, EETQ). Powers Hugging Face Inference Endpoints and is the default backend when deploying models from the Hub. Tight integration with the transformers library makes it the path of least resistance for HF users going to production.

Latest release v3.3.7 (Dec 19, 2025). Repo ARCHIVED Mar 21, 2026 and placed in maintenance mode; HF now recommends vLLM/SGLang/llama.cpp/MLX going forward. Verified live (archived) June 2026.

Openness

5 high confidence

5.0

license: Apache-2.0(OSI)
source: public
note: was briefly under restrictive HFOIL license in 2023, reverted to Apache-2.0

Current LICENSE is standard Apache-2.0 (not HFOIL); fully open source despite being archived.

https://github.com/huggingface/text-generation-inference/blob/main/LICENSE recorded 2026-06-04
Apache License 2.0 text, Copyright 2022 Hugging Face

Adoption

3 medium confidence

3.0

Ran in production at HF (Hugging Chat, Inference API, Inference Endpoints) and was widely deployed 2023-2025, but the repo is now archived/maintenance-mode with HF steering users to vLLM/SGLang, so adoption is declining. 10.9k stars (secondary). Level 3 reflects a real but shrinking, sunsetting footprint.

https://github.com/huggingface/text-generation-inference recorded 2026-06-04
README: 'Used in production at Hugging Face to power Hugging Chat, the Inference API and Inference Endpoints'; archived Mar 21 2026, maintenance mode, recommends vLLM/SGLang

Capability

3 medium confidence

3.0

Was a capable production server, but development has stopped (archived); its feature set is frozen while vLLM/SGLang continue advancing, so it now sits mid-pack rather than at the C5 frontier.

https://github.com/huggingface/text-generation-inference recorded 2026-06-04
README documents continuous batching, tensor parallelism, OpenTelemetry tracing, Prometheus metrics, production-ready feature set

Unchanged since 2026-06-09 (last edited, not re-checked)