Back to Gap Map Model components / Inference code

LiteRT

Google
open source / Overall score: 3.5

LiteRT is Google's on-device runtime for machine learning and generative AI, and the successor to TensorFlow Lite. It runs converted PyTorch, TensorFlow and JAX models on Android, iOS, the web, desktop and IoT devices, choosing a CPU, GPU or NPU accelerator automatically, with NPUs from Google Tensor, Qualcomm, MediaTek, Intel and others behind one API. LiteRT-LM, which runs language models on device, is built on it.

TensorFlow Lite was renamed LiteRT, and the .tflite model format carries over. LiteRT-LM, the language-model layer built on this runtime, is its own product on the map.

Openness

5 high confidence
5.0
license
Apache-2.0 (OSI)
source
public
core features withheld
no

LiteRT is Apache-2.0 licensed, and the repository holds the runtime, its accelerator layers and the guide for building an NPU backend outside it. A chipmaker's own NPU SDK, such as Qualcomm's, carries that vendor's terms, but nothing in LiteRT itself is gated.

Adoption

3 medium confidence
3.0

PyPI downloads of the two Python packages, ai-edge-litert and tflite-runtime, the TensorFlow Lite runtime package it replaced, which still sees heavy use. Android apps, LiteRT's main audience, take it from Google's Maven repository, and Maven publishes no download count, so the Android channel is left out and this is a floor on use.

Capability

4 medium confidence
4.0

LiteRT sits a tier below vLLM, level with ExecuTorch, its closest peer: both run converted models on phone, desktop and embedded silicon, and LiteRT's single NPU API covers a comparable spread of chip vendors. It runs models on a device rather than serving them with batching, parallelism and a multi-tenant API.

  • https://raw.githubusercontent.com/google-ai-edge/LiteRT/main/README.md recorded 2026-11-12

    The README lists the "Compiled Model API" with "automated accelerator selection" and "true asynchronous execution", "Unified NPU Acceleration", "GPU Acceleration via ML Drift", CPU via XNNPack, and a platform table covering Android (Broadcom, Google Tensor, Intel, MediaTek, Qualcomm, S.LSI NPUs), iOS, Linux, macOS, Windows, Web and IoT.

Verified 2026-11-12