Set up AI Observability

Learn about the high-level steps to set up Splunk AI Observability.

Attention: Beta features described in this document are provided by Splunk to you "as is" without any warranties, maintenance and support, or service-level commitments. Splunk makes this beta feature available in its sole discretion and may discontinue it at any time. Use of beta features is subject to the Splunk General Terms.

Monitor and troubleshoot your AI components by sending data from AI components to Splunk Observability Cloud.

Complete the following high-level steps to set up and use AI Observability.

  1. Collect metrics and metadata from AI components.
  2. Collect traces and logs from AI components.
  3. Monitor and troubleshoot your AI components.

Collect metrics and metadata from AI components

Learn how to collect metrics and metadata for Splunk AI Observability.

Splunk Observability Cloud supports multiple data ingestion and connection methods to collect your Amazon Web Services (AWS), Azure, and Google Cloud Platform metrics and metadata. To collect metrics and metadata from all other AI components, you must install the Splunk Distribution of the OpenTelemetry Collector and configure an OpenTelemetry receiver.

To collect metrics and metadata, refer to the following documentation for your AI component:

Collect traces and logs from AI components

Learn how to collect traces and logs from AI components for Splunk AI Observability.

Splunk Observability Cloud uses the Splunk HTTP Event Collector (HEC) exporter to enable the Splunk Distribution of the OpenTelemetry Collector to collect traces and logs. Splunk Log Observer Connect correlates the logs with metrics and traces for advanced troubleshooting.

To collect traces and logs for your AI components, complete the following high-level steps:

  1. Set up Log Observer Connect for Splunk Cloud Platform.

  2. Configure the Splunk HEC exporter to send logs to Splunk Cloud Platform.

Monitor and troubleshoot your AI components

Learn about the tools you can use to monitor and troubleshoot your AI components in Splunk AI Observability.

After you set up data collection from supported AI components to Splunk Observability Cloud, the data populates built-in experiences that you can use to monitor and troubleshoot your AI components.

The following table describes the tools you can use to monitor and troubleshoot your AI components.
Monitoring toolUse this tool toLink to documentation
Built-in navigatorsOrient and explore different layers of your AI tech stack.
Built-in dashboardsAssess service, endpoint, and system health at a glance.
Splunk Application Performance Monitoring (APM) service map and trace viewView all of your LLM service dependency graphs and user interactions in the service map or trace view.

Monitor LLM services with Splunk APM