Vertex AI Model Observability
Google CloudGoogle Cloud's production monitoring surface for generative AI on Vertex AI: a predefined Cloud Monitoring dashboard tracking usage, throughput, token consumption, latency and error rates, including capacity errors, for Gemini and other Model Garden foundation models. Vertex AI Agent Builder extends it with sessions, traces, logs, agent performance dashboards, multi-turn auto-raters, online evaluation against live traffic and a unified trace viewer for debugging agent reasoning.
Google publishes no usage figure for this surface specifically, so its adoption band leans on the reach of Vertex AI rather than on a measurement of the surface. Verified 2026-08-13 via the Agent Engine tracing documentation.
Openness
1 high confidence- license
- proprietary(Google Cloud managed service)
- source
- closed
- note
- tracing is built on the open OTel standard but the observability product/service itself is proprietary and cloud-locked
Proprietary GCP managed offering (Agent Engine + Cloud Trace/Monitoring/Logging). OTel is the wire standard but the service is closed and tied to Google Cloud.
- https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-08-13
Documentation - "Cloud Trace lets you analyze the performance of your agents by tracing" them, with "A trace is composed of individual spans, which represent a single unit of work, like a function call or an interaction with an LLM." The trace format shown carries trace_id, span_id and span_kind LLM. Everything described runs inside Google Cloud; nothing is published to run elsewhere.
Adoption
3 low confidenceA feature surface within Vertex AI Agent Engine on Google Cloud; no standalone usage figures published for the observability capability specifically. Level 3 reflects availability inside a large cloud platform's agent stack, not a measured user count; directional. The quantity actually banded is the reach of Vertex AI rather than usage of its tracing surface, and the documentation discloses no figure for either.
- https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-08-13
The tracing documentation, read for a usage figure and containing none. It sits under an Observability section of the Agent Engine docs, beside RAG Engine deployment modes and Model Armor, which places the capability inside the platform without saying how many people use it.
Capability
3 medium confidenceSolid tracing, monitoring and integrated evaluation: traces and spans for LLM calls, tool calls and agent decisions through Cloud Trace and OpenTelemetry; a performance dashboard covering token consumption, latency, error rates and tool calls; the Gen AI Evaluation service including its User Simulator; and Cloud Logging. Cost and token tracking are present, but prompt management and dedicated annotation queues are not evidenced as native, which makes the matrix thinner than the dedicated open source platforms and puts this a rung below them at 3.
- https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-08-13
Documentation covering Cloud Trace spans for ADK agents, with span_kind LLM, parent_id and event fields shown in the example payload, and an Observability section grouping it with monitoring. No prompt-versioning or annotation-queue feature is described.
Verified 2026-08-13