Model components / Inference code

Google Cloud TPU Inference

Proprietary inference runtime for Google's Tensor Processing Units (TPUs), now in v5e/v6e generations. The Pathways system enables efficient model serving across TPU pods with automatic model partitioning. Powers Google's own production inference for Gemini and internal models. Available via Vertex AI and GKE for external customers. The most mature custom-silicon inference stack, developed over 8+ years of TPU generations.

Pathways runtime (Google DeepMind distributed runtime) made available on Google Cloud and integrated into JetStream for multi-host/disaggregated TPU serving (2026, AI Hypercomputer). Proprietary managed cloud offering on Trillium v6 / Ironwood TPUs. Confirmed live June 2026.

Openness

1 high confidence

1.0

license: proprietary
runtime: Pathways(Google-internal distributed runtime, not open sourced)
hardware-lock: Google TPU-only
managed: Google Cloud/GKE
note: JetStream(separate, Apache-2.0 OSS) is the open serving frontend but the scored product is the proprietary Pathways/TPU runtime

The named product (Pathways / TPU Runtime managed inference) is proprietary, TPU-locked, and cloud-delivered. JetStream (open) is a distinct artifact and not what this SKU is; scored closed.

https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu recorded 2026-06-04
Pathways runtime offered as managed Google Cloud component (not labeled open source); JetStream described as the open source engine separately

Adoption

3 low confidence

3.0

Powers Google internal + Cloud TPU inference via AI Hypercomputer/GKE; no standalone usage figure for the Pathways runtime offering. Level 3 = reported managed-cloud traction.

https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu recorded 2026-06-04
Pathways/JetStream managed TPU inference on AI Hypercomputer, GKE-integrated, 3-yr CUD pricing

Capability

4 medium confidence

4.0

High-throughput multi-host TPU inference (1703 tok/s Llama 405B on Trillium, primary Google figure). Strong but TPU-locked and not an independent MLPerf submission for the runtime; C4, below the open frontier engines' breadth.

https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu recorded 2026-06-04
1703 tok/s Llama 3.1 405B on Trillium, ~3x inference/$ vs v5e

Unchanged since 2026-06-24 (last edited, not re-checked)