Monitor LLM costs with navigators

Learn how to monitor LLM costs with navigators.

Note: This feature is only supported on the LangChain and OpenAI navigators.
Splunk Observability Cloud collects OpenTelemetry generative AI client and model server metrics to track the token usage and costs of large-language model (LLM) services. On supported navigators, you can use metric charts to track the cost and token usage for LLM services and models.

Complete the following steps to monitor LLM costs with navigators.

  1. From the Splunk Observability Cloud main menu, select Infrastructure > Overview.
  2. Under AI/ML, select AI Frameworks.
  3. Select the LangChain or OpenAI navigator.
  4. Use one of the following methods to monitor LLM costs:
    • The Overview tab is selected by default. View the charts for cost and token usage.
    • Select the Table tab, then the name of a service to display the detail view. View the charts for cost and token usage.
    The following screenshot displays the detail view of the LangChain navigator, which includes metric charts for cost and token usage.A screenshot of the detail view for the LangChain navigator, which contains metric charts that you can use to monitor cost and token usage.