Collect metrics and traces from LangChain services

Learn how to collect metrics and traces from LangChain services.

You can collect metrics and traces from LangChain large-language model (LLM) services by instrumenting your Python application using the Splunk Distribution of OpenTelemetry Python with the OpenTelemetry LangChain instrumentation. The LangChain instrumentation pushes metrics and traces to the OTLP receiver.

Complete the following steps to collect metrics and traces from LangChain services and monitor their performance using navigators.

(Prerequisite) Initialize the LangChain instrumentation in your service.
Configure and activate the component for LangChain.
Use the LangChain navigator to monitor the performance of LangChain services.

Prerequisites

Learn how to initialize the LangChain instrumentation in your service.

To use the LangChain instrumentation to push LangChain metrics and traces to the OTLP receiver, you must install and initialize the LangChain instrumentation in your service.

Run the following command:

pip install git+https://github.com/zhirafovod/opentelemetry-python-contrib.git@instrumentation-genai-langchain#subdirectory=instrumentation-genai/opentelemetry-instrumentation-langchain

Add the following code to your service:

from opentelemetry.instrumentation.langchain import
LangChainInstrumentor
LangChainInstrumentor().instrument()

Configure and activate the component for LangChain

Learn how to configure and activate the component to monitor LangChain services.

Complete the following steps to configure and activate the component for LangChain.

Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:
To activate the OTLP receiver for LangChain manually in the Collector configuration, add the OpenTelemetry receiver address to the configuration of your LangChain service. This step is required to send telemetry to the receiver. For example:
```
otel_endpoint = os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT")
```

Add the following environment variables to the configuration of your LangChain service:

OTEL_RESOURCE_ATTRIBUTES="deployment.environment=<environment_name>"
OTEL_SERVICE_NAME=<service_name>
OTEL_INSTRUMENTATION_LANGCHAIN_CAPTURE_MESSAGE_CONTENT=<true_or_false>
OPENAI_API_KEY=<openai_api_key>

Deploy the Python agent in your LangChain service:
1. Install the Splunk Distribution of OpenTelemetry Python using the guided setup or manual method.
2. Install the AI/LLM instrumentation by following the steps on LangChain instrumentation in the OpenTelemetry Python Contrib GitHub repository. You can use zero-code instrumentation or manual instrumentation.
Run your service.

Monitor the performance of LangChain services

Learn how to navigate to the LangChain navigator, which you can use to monitor the performance of LangChain services.

Complete the following steps to access the LangChain navigator and monitor the performance of LangChain services. For more information on navigators, see Use navigators and Monitor LLM costs with navigators.

From the Splunk Observability Cloud main menu, select Infrastructure.
Under AI/ML, select AI Frameworks.
Select the LangChain summary card.

Configuration settings

Learn about the configuration settings for the OTLP receiver.

To view the configuration options for the OTLP receiver, see Settings.

Metrics

Learn about the monitoring metrics available for LangChain services.

The following metrics are available for LangChain services. For more information, see Semantic conventions for generative AI metrics in the OpenTelemetry documentation.

Metric name	Type	Unit	Description
`gen_ai.client.operation.duration`	float	s	The duration of the GenAI operation.
`gen_ai.client.token.usage`	int	count	Measures the number of input (prompt) and output (completions) tokens used.

Attributes

Learn about the resource attributes available for LangChain services.

The following resource attributes are available for LangChain services.

OpenTelemetry attribute name	Description
`telemetry.sdk.name`	The name of the telemetry SDK.
`telemetry.sdk.language`	The language of the telemetry SDK.
`telemetry.sdk.version`	The version string of the telemetry SDK.
`deployment.environment`	The environment where the GenAI service is running.
`service.name`	The name of the service.
`host.name`	The name of the host.
`os.type`	The OS type.

Troubleshoot

Learn how to get help if you can't see your data in Splunk Observability Cloud.

If you can't see your data in Splunk Observability Cloud, you can get help in the following ways:

Splunk Observability Cloud customers can submit a case in the Splunk Support Portal or contact Splunk Support.

Prospective customers and free trial users can ask a question and get answers through community support in the Splunk Community.

Documentation