Configure code-based instrumentation for Python AI applications

Instrument your backend Python AI applications to send metrics, traces, and evaluations to Splunk Observability Cloud.

Instrument your backend Python AI applications to send traces and metrics to Splunk Observability Cloud.

Code-based instrumentation requires modifying your application's code using the Splunk Distribution for OpenTelemetry Generative AI utility. The code-based instrumentation provides APIs and GenAI data types to simplify your GenAI application instrumentation. This ensures your GenAI telemetry follows semantic conventions and can be processed by Splunk Observability Cloud.

To instrument a Python AI application, you must meet the following requirements.

  • If your application is already instrumented with an OpenTelemetry SDK, you must upgrade your OpenTelemetry dependencies to version 1.38.0 or higher.

  • You have installed the Splunk Distribution for OpenTelemetry Generative AI packages with the following command:
    CODE 
    pip install splunk-otel-util-genai
  1. Refer to GenAI Types (Data Model) in the splunk-otel-python-contrib GitHub repository to identify which GenAI data types are relevant for your application. The core data types are:
    • Workflow: A logical grouping of multiple GenAI operations that represent a high-level orchestration. Usually a root GenAI span.
    • AgentInvocation: One logical agent that groups multiple GenAI operations. Represented by the invoke_agent operation.
    • LLMInvocation: An LLM model call.
    • InputMessage, OutputMessage: The structured messages used to communicate with an LLM. Consists of multiple part objects. A part object is a pair that contains a role and message. Each part can be a Text, image, or other type.
  2. Instrument your application by creating the required invocations for your data types.
    The following custom instrumentation example uses Workflow, AgentInvocation, and LLMInvocation to instrument the code of an AI application.
    PYTHON 
    from opentelemetry.util.genai.types import (
    Workflow,
    AgentInvocation,
    LLMInvocation,
    InputMessage,
    OutputMessage,
    Text,
)
from opentelemetry.util.genai.handler import get_telemetry_handler
from openai import OpenAI

handler = get_telemetry_handler()
workflow = Workflow(name="agent_demo", workflow_type="planner", input_messages=[InputMessage(role="user", parts=[Text(content="Plan a 2-day trip to Rome")])])
handler.start_workflow(workflow)

agent = AgentInvocation(
    name="trip_planner",
    agent_type="planner",
    model="gpt-4o-mini",
    system_instructions="You plan concise city itineraries",
    input_messages=workflow.input_messages,
)
handler.start_agent(agent)

llm_call = LLMInvocation(
    request_model="gpt-4o-mini",
    operation="chat",
    input_messages=[
        InputMessage(role="system", parts=[Text(content="You provide day-by-day plans.")]),
        InputMessage(role="user", parts=[Text(content="Plan a 2-day trip to Rome focusing on food and history.")]),
    ],
)
llm_call.provider = "openai"
llm_call.framework = "native-client"
handler.start_llm(llm_call)

client = OpenAI()
openai_messages = [
    {"role": m.role, "content": "".join(p.content for p in m.parts if hasattr(p, "content"))}
    for m in llm_call.input_messages
]
response = client.chat.completions.create(
    model=llm_call.request_model,
    messages=openai_messages,
    temperature=0.3,
)

choice = response.choices[0]
assistant_text = choice.message.content
llm_call.output_messages = [
    OutputMessage(role="assistant", parts=[Text(content=assistant_text)], finish_reason=choice.finish_reason or "stop")
]
if response.usage:
    llm_call.input_tokens = response.usage.prompt_tokens
    llm_call.output_tokens = response.usage.completion_tokens

agent.output_result = assistant_text
handler.stop_llm(llm_call)
handler.stop_agent(agent)
workflow.final_output = assistant_text
handler.stop_workflow(workflow)
    For more examples, see the In-House Instrumentation Example: Multi-Agent Travel Planner and the main_in_house.py file in the splunk-otel-python-contrib GitHub repository.

To finish setting up AI Agent Monitoring, proceed to the next step in Set up AI Agent Monitoring.