Back to Gap Map Product / UX / Telemetry & observability

Datadog LLM Observability

Datadog
closed / Overall score: 4.0(strong)

Enterprise LLM monitoring integrated into the Datadog APM platform. Traces LLM calls alongside existing infrastructure, application, and security monitoring. For organizations already using Datadog who want unified observability across their entire stack including AI/LLM components. Covers agent monitoring, prompt tracking and management, quality monitoring, and correlation of LLM spans with APM.

Datadog publishes no usage figure for this module, so its adoption reading stands on the reach of the Datadog platform rather than on a measurement of the module.

Openness

1 high confidence
1.0
license
Proprietary
source
closed
platform
proprietary SaaS(no source)

Closed proprietary SaaS. Datadog's subscription terms give it all rights in the service and forbid recreating its source code, while the ddtrace client library that sends data to it is open under Apache 2.0 or BSD-3-Clause.

  • https://docs.datadoghq.com/llm_observability/instrumentation/sdk/ recorded 2026-09-29

    LLM Observability SDK setup page, showing the open ddtrace client library reporting to the hosted service, with the setting "Your Datadog API key. Only required if you are not using the Datadog Agent".

  • https://raw.githubusercontent.com/DataDog/dd-trace-py/main/LICENSE recorded 2026-09-29

    License file of the ddtrace Python tracing library, stating "This work is dual-licensed under Apache 2.0 or BSD3." with the identifier "SPDX-License-Identifier: Apache-2.0 OR BSD-3-Clause".

  • https://www.datadoghq.com/legal/msa/ recorded 2026-09-29

    Datadog Master Subscription Agreement, which states that "Datadog owns all right, title and interest in and to the Services" and bars customers from any attempt to "reverse engineer, disassemble or decompile all or any portion of, or attempt to access, discover or recreate the source code for, the Services".

Adoption

4 low confidence
4.0

LLM Observability rides the large installed base of Datadog APM, a major enterprise observability vendor, but it is a newer add-on with no disclosed standalone user or usage count of any kind in the docs. The level is therefore attributed to the parent platform and is directional: what is banded is the reach of Datadog overall, not usage of the LLM Observability module.

  • https://docs.datadoghq.com/llm_observability/ recorded 2026-08-13

    The module's documentation, read for a usage figure and containing none - no customer count, no account count, no ingested-span volume. It sits inside the Datadog docs beside APM, Log Management, Cloud SIEM and the rest of the platform, which is the only thing on the page that speaks to reach.

Capability

4 medium confidence
4.0

Datadog LLM Observability covers end-to-end trace capture, built-in and custom evaluations, cost and usage dashboards, sensitive-data scanning and prompt-injection detection, and topic clustering of production traffic, integrated with the wider Datadog APM suite and auto-instrumented for major LLM frameworks. That breadth keeps it level with Langfuse.

  • https://docs.datadoghq.com/llm_observability/ recorded 2026-09-29

    Docs landing page: "Monitor and improve model performance. Pinpoint root causes and detect anomalies" and "Automatically scan and redact any sensitive data in your AI applications and identify prompt injections, among other evaluations." with Quality Monitoring, Prompt Tracking and Correlate with APM in the navigation. Searched for a customer or usage count: none.

  • https://docs.datadoghq.com/llm_observability/instrumentation/sdk/ recorded 2026-09-29

    SDK page on cost: "Datadog supports common model providers such as OpenAI, Azure OpenAI, Anthropic, and Google Gemini." and it shows LangChain prompt templates picked up by auto-instrumentation.

  • https://docs.datadoghq.com/llm_observability/monitoring/ recorded 2026-09-29

    Investigate page: "This feature helps you track agent actions, tool usage, and reasoning steps, providing visibility into complex LLM workflows" and shows a Hallucination evaluation on a span.

  • https://www.datadoghq.com/product/llm-observability/ recorded 2026-09-29

    Product page, now titled Agent Observability: "Catch hallucinations, prompt injection attempts, and PII exposure as they happen." Searched for a customer count or usage figure for the module: none.

Verified 2026-09-29, except some scores, still awaiting re-confirmation