Configure your Splunk Observability Cloud account to collect Azure OpenAI metrics

Learn how to configure your Splunk Observability Cloud account to collect Azure OpenAI metrics.

You can monitor the performance of Azure OpenAI applications by configuring your Azure OpenAI applications to send metrics to Splunk Observability Cloud. This solution creates a cloud connection in your Splunk Observability Cloud account that collects metrics from Azure application insights.

Complete the following steps to collect metrics from and monitor your Azure OpenAI applications.

  1. Connect Azure to Splunk Observability Cloud. For more information on the connection methods and instructions for each method, see Available options to connect with Azure.
    Note: If you connect with Azure using the guided setup, the Select the data to import page includes the option to import data from All Azure services or Specified Azure services only. If you select Specified Azure services only, select Cognitive Services from the Specified services drop-down menu to ensure that Azure OpenAI metrics are ingested.
  2. Run your app.py file.
    You call your Azure OpenAI models within your application either on local or on cloud. The following command calls a local Python application file with model invocations.
    python app.py
  3. Use the Azure OpenAI navigator to monitor your Azure OpenAI applications. For more information on navigators, see Use navigators.
    1. From the Splunk Observability Cloud main menu, select Infrastructure.
    2. Under AI/ML, select AI Frameworks.
    3. Select the Azure OpenAI summary card.

Metrics and attributes

Learn about the available metrics for Azure OpenAI.

The following metrics and resource attributes are available for Azure OpenAI applications.
Metric nameTypeUnitDescriptionDimensions
ProcessedPromptTokenscountercountThe number of prompt tokens processed (input) on an OpenAI model.

  • ApiName

  • ModelDeploymentName

  • FeatureName

  • UsageChannel

  • Region

  • ModelVersion

GeneratedTokens

counter

countThe number of tokens generated (output) on an OpenAI model.

  • ApiName

  • ModelDeploymentName

  • FeatureName

  • UsageChannel

  • Region

  • ModelVersion

AzureOpenAIRequestscountercountThe number of calls made to the Azure OpenAI service.

  • ApiName

  • OperationName

  • Region

  • ModelDeploymentName

  • ModelName

  • ModelVersion

  • StatusCode

AzureOpenAITimeToResponsehistogrammsThe time taken for the first response to appear after the user sends a prompt.

  • ApiName

  • OperationName

  • Region

  • ModelDeploymentName

  • ModelName

  • ModelVersion

  • StatusCode

AzureOpenAIAvailabilityRategaugepercentThe availability percentage with the following calculation: (Total Calls - Server Errors)/Total Calls. Server Errors include any HTTP responses >=500.

  • ApiName

  • OperationName

  • Region

  • ModelDeploymentName

  • ModelName

  • ModelVersion

  • StatusCode

AzureOpenAITokenPerSecond

gaugecountThe generation speed for a given Azure OpenAI model response.

  • Region

  • ModelDeploymentName

  • ModelName

  • ModelVersion

AzureOpenAIContextTokensCacheMatchRategaugepercentThe percentage of prompt tokens that hit the cache.

  • Region

  • ModelDeploymentName

  • ModelName

  • ModelVersion

Troubleshoot

Learn how to get help if you can't see your data in Splunk Observability Cloud.

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways:

  • Prospective customers and free trial users can ask a question and get answers through community support in the Splunk Community.