Datadog LLM Observability Is Now Generally Available to Help Businesses Monitor, Improve and Secure Generative AI Applications

News provided by

Jun 26, 2024, 10:00 ET

New product helps companies like WHOOP and AppFolio monitor hallucinations, adopt LLMs and release generative AI features with confidence

NEW YORK, June 26, 2024 /PRNewswire/ -- Datadog, Inc. (NASDAQ: DDOG), the monitoring and security platform for cloud applications, today announced the general availability of LLM Observability, which allows AI application developers and machine learning (ML) engineers to efficiently monitor, improve and secure large language model (LLM) applications. With LLM Observability, companies can accelerate the deployment of generative AI applications to production environments and scale them reliably.

Organizations across all industries are racing to release generative AI features in a cost-effective way, but implementing and bringing them to production can present several challenges due to the complexity of LLM chains, their non-deterministic nature and the security risks they pose.

Datadog LLM Observability helps customers overcome these challenges so they can confidently deploy and monitor their generative AI applications. This new product provides visibility into each step of the LLM chain to easily identify the root cause of errors and unexpected responses such as hallucinations. Users can also monitor operational metrics like latency and token usage to optimize performance and cost, and can evaluate the quality of their AI applications—such as topic relevance or toxicity—and gain insights to mitigate security and privacy risks with out-of-the-box quality and safety evaluations.

Unlike traditional tools and point solutions, Datadog's LLM Observability offers prompt and response clustering, seamless integration with Datadog Application Performance Monitoring (APM), and out-of-the-box evaluation and sensitive data scanning capabilities to enhance the performance, accuracy and security of generative AI applications while helping to keep data private and secure.

"WHOOP Coach is powered by the latest and greatest in LLM AI. Datadog's LLM Observability allows our engineering teams to evaluate performance of model changes, monitor production performance and increase quality of Coach interactions. LLM Observability allows WHOOP to provide and maintain coaching for all our members 24/7," said Bobby Johansen, Senior Director Software at WHOOP.

"The Datadog LLM Observability solution helps our team understand, debug and evaluate the usage and performance of our GenAI applications. With it, we are able to address real-world issues, including monitoring response quality to prevent negative interactions and performance degradations, while ensuring we are providing our end users with positive experiences," said Kyle Triplett, VP of Product at AppFolio.

"There's a rush to adopt new LLM-based technologies, but organizations of all sizes and industries are finding it difficult to do so in a way that is both cost effective and doesn't negatively impact the end user experience," said Yrieix Garnier, VP of Product at Datadog. "Datadog LLM Observability provides the deep visibility needed to help teams manage and understand performance, detect drifts or biases, and resolve issues before they have a significant impact on the business or end-user experience."

LLM Observability helps organizations:

Evaluate Inference Quality: Visualize the quality and effectiveness of LLM applications' conversations—such as failure to answer—to monitor any hallucinations, drifts and the overall experience of the apps' end users.
Identify Root Causes: Quickly pinpoint the root cause of errors and failures in the LLM chain with full visibility into end-to-end traces for each user request.
Improve Costs and Performance: Efficiently monitor key operational metrics for applications across all major platforms—including OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Vertex AI and more—in a unified dashboard to uncover opportunities for performance and cost optimization.
Protect Against Security Threats: Safeguard applications against prompt hacking and help prevent leaks of sensitive data, such as PII, emails and IP addresses, using built-in security and privacy scanners powered by Datadog Sensitive Data Scanner.

Datadog LLM Observability is generally available now. To learn more, please visit: http://datadoghq.com/product/llm-observability.

About Datadog

Datadog is the observability and security platform for cloud applications. Our SaaS platform integrates and automates infrastructure monitoring, application performance monitoring, log management, user experience monitoring, cloud security and many other capabilities to provide unified, real-time observability and security for our customers' entire technology stack. Datadog is used by organizations of all sizes and across a wide range of industries to enable digital transformation and cloud migration, drive collaboration among development, operations, security and business teams, accelerate time to market for applications, reduce time to problem resolution, secure applications and infrastructure, understand user behavior and track key business metrics.

Forward-Looking Statements

This press release may include certain "forward-looking statements" within the meaning of Section 27A of the Securities Act of 1933, as amended, or the Securities Act, and Section 21E of the Securities Exchange Act of 1934, as amended including statements on the benefits of new products and features. These forward-looking statements reflect our current views about our plans, intentions, expectations, strategies and prospects, which are based on the information currently available to us and on assumptions we have made. Actual results may differ materially from those described in the forward-looking statements and are subject to a variety of assumptions, uncertainties, risks and factors that are beyond our control, including those risks detailed under the caption "Risk Factors" and elsewhere in our Securities and Exchange Commission filings and reports, including the Quarterly Report on Form 10-Q filed with the Securities and Exchange Commission on November 7, 2023, as well as future filings and reports by us. Except as required by law, we undertake no duty or obligation to update any forward-looking statements contained in this release as a result of new information, future events, changes in expectations or otherwise.

Contact
Dan Haggerty
[email protected]

SOURCE Datadog, Inc.