Back to Gap Map Model components / Inference code

CTranslate2

OpenNMT
open source / Overall score: 4.0(strong)

CTranslate2 is a C++ and Python inference engine for Transformer models that speeds them up and cuts their memory use on CPUs and GPUs through weight quantization, layer fusion and batch reordering. It runs converted encoder-decoder, decoder-only and encoder-only models, including Whisper, NLLB, T5 and Llama-family models, and is the engine behind faster-whisper. It is part of the OpenNMT project.

Openness

5 high confidence
5.0
license
MIT (OSI)
source
public (the engine, Python bindings and model converters)
core features withheld
no — part of the open-source OpenNMT project with no paid edition and no enterprise path in the repository tree

CTranslate2 is MIT-licensed, and the engine and its model converters build from the public repository. It belongs to the open-source OpenNMT project and has no paid edition, so nothing is held back.

Adoption

5 high confidence
5.0

PyPI downloads of the ctranslate2 package, which carries the compiled engine. Part of the count comes in as a dependency of libraries built on it, such as faster-whisper, and each of those still installs the engine.

Capability

3 medium confidence
3.0

CTranslate2 is a fast, light engine for running Transformer models on CPUs and GPUs, used above all for translation and speech models. It is a library rather than a serving system and documents none of the request-level techniques vLLM is built on, such as paged attention and continuous batching, so it sits two steps below vLLM.

  • https://raw.githubusercontent.com/OpenNMT/CTranslate2/HEAD/README.md recorded 2026-11-12

    The README lists "weights quantization, layers fusion, batch reordering", FP16, BF16, INT16, INT8 and AWQ (INT4) weights, Intel MKL, oneDNN, OpenBLAS, Ruy and Apple Accelerate CPU backends with automatic dispatch, "Support tensor parallelism for distributed inference", and supported encoder-decoder, decoder-only and encoder-only model types.

Verified 2026-11-12