Configure the OTLP receiver to collect OpenAI metrics

Learn how to configure the OTLP receiver to collect OpenAI metrics.

You can monitor the performance of OpenAI services by configuring your OpenAI services to send metrics to Splunk Observability Cloud. This solution uses the OpenTelemetry instrumentation to collect metrics from OpenAI services.

Complete the following steps to collect metrics from and monitor the performance of OpenAI services.

  1. Configure and activate the component for OpenAI.
  2. Use the OpenAI navigator to observe the performance of OpenAI services.

Configure and activate the component for OpenAI

Learn how to configure and activate the component for OpenAI services.

Complete the following steps to configure and activate the component for OpenAI.

  1. Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:
  2. Start the Splunk Distribution of the OpenTelemetry Collector.
  3. Deploy the Python agent in your OpenAI service:
    1. Install the Splunk Distribution of OpenTelemetry Python using the guided setup or manual method.
    2. Install the Generative AI/LLM instrumentation by following the steps on OpenTelemetry OpenAI Instrumentation in the OpenTelemetry Python Contrib GitHub repository. You can use zero-code instrumentation or manual instrumentation.
  4. Run the service.

Monitor the performance of OpenAI services

Learn how to navigate to the OpenAI navigator, which you can use to observe the performance of OpenAI services.

Complete the following steps to access the OpenAI navigator and monitor the performance of OpenAI services. For more information on navigators, see Use navigators and Monitor LLM costs with navigators.

  1. From the Splunk Observability Cloud main menu, select Infrastructure.
  2. Under AI/ML, select AI Frameworks.
  3. Select the OpenAI summary card.

Metrics and attributes

Learn about the monitoring metrics available for OpenAI services.

The following metrics and resource attributes are available for OpenAI services. For more information, see Metrics in the OpenTelemetry documentation.
Metric nameHistogram functionInstrument typeUnitDescriptionResource attributes
gen_ai.client.token.usageinputhistogramcountNumber of input tokens processed.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model error.type
gen_ai.client.token.usageoutput histogramcountNumber of output tokens processed.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.token.usagesumhistogramcountSum of tokens processed.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.token.usagemaxhistogramcountMax number of total tokens processed.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.token.usageminhistogramcountMin number of total tokens processed.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.operation.durationsumhistogramsTotal time of durations.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.operation.durationmaxhistogramsMax length of durations.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.operation.durationminhistogramsMin length of durations.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type
gen_ai.client.operation.durationcounthistogramcountCount of durations.
  • server.port
  • server.address
  • gen_ai.application_name
  • gen_ai.system
  • gen_ai.environment
  • gen_ai.operation.name
  • gen_ai.request.model
  • gen_ai.response.model
  • error.type

Troubleshoot

Learn how to get help if you can't see your data in Splunk Observability Cloud.

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways:

  • Prospective customers and free trial users can ask a question and get answers through community support in the Splunk Community.