Vertex AI Model Observability

license

proprietary(Google Cloud managed service)

source

closed

note

tracing is built on the open OTel standard but the observability product/service itself is proprietary and cloud-locked

Proprietary GCP managed offering (Agent Engine + Cloud Trace/Monitoring/Logging). OTel is the wire standard but the service is closed and tied to Google Cloud.

• https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-09-29

  Tracing page: "Cloud Trace lets you analyze the performance of your agents by tracing the timeline of operations for each query." and traces are enabled by deploying the agent to Agent Runtime with "GOOGLE_CLOUD_AGENT_ENGINE_ENABLE_TELEMETRY" set. Searched the body for self-host, on-prem, open source and license: none.

• https://docs.cloud.google.com/vertex-ai/generative-ai/docs/learn/model-observability recorded 2026-09-29

  Monitor models page: Agent Platform "provides a prebuilt model observability dashboard to view the behavior, health, and performance of fully-managed models" that are "provided by Google", with metrics "such as model request per second (QPS), token throughput, and first token latencies". Searched the body for on-prem and open source: none.

Vertex AI Model Observability is a feature surface within Vertex AI Agent Engine on Google Cloud, and Google publishes no separate usage figure for it. The reading instead reflects the reach of Vertex AI as a platform, not a measured count of people using this observability capability itself.

• https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-08-13

  The tracing documentation, read for a usage figure and containing none. It sits under an Observability section of the Agent Engine docs, beside RAG Engine deployment modes and Model Armor, which places the capability inside the platform without saying how many people use it.

Vertex AI Model Observability provides tracing through Cloud Trace and OpenTelemetry, a prebuilt dashboard covering request rate, token throughput, latency and error rates, agent metrics, and online monitors that score live traces. Prompt management and dedicated annotation queues are not documented as native features, which leaves it a step behind Langfuse.

• https://docs.cloud.google.com/agent-builder/agent-engine/manage/tracing recorded 2026-09-29

  Tracing page: "Cloud Trace lets you analyze the performance of your agents by tracing the timeline of operations for each query." and spans carry "span_kind" LLM with input and output messages. Searched the body for prompt management and annotation queue: none.

• https://docs.cloud.google.com/gemini-enterprise-agent-platform/optimize/evaluation/evaluate-online recorded 2026-09-29

  Online monitoring page: "Online monitoring lets you continuously assess the quality of your agents in production." and it requires Cloud Trace plus OpenTelemetry signals from a deployed agent.

• https://docs.cloud.google.com/gemini-enterprise-agent-platform/scale/runtime/monitoring recorded 2026-09-29

  Agent Runtime monitoring page: "Agent Runtime automatically collects and processes operational metrics for deployed agents." with built-in metrics for "Request count" and "Request latencies", plus alerts and PromQL queries.

• https://docs.cloud.google.com/vertex-ai/generative-ai/docs/learn/model-observability recorded 2026-09-29

  Monitor models page: Agent Platform "provides a prebuilt model observability dashboard to view the behavior, health, and performance of fully-managed models" that are "provided by Google", with metrics "such as model request per second (QPS), token throughput, and first token latencies". Searched the body for on-prem and open source: none.