Install the Splunk Python agent

Get started with the Splunk Python agent.

Note: Due to changes in the upstream OpenTelemetry documentation, "automatic instrumentation" has been changed to "zero-code instrumentation". For more information, see Instrumentation methods for Splunk Observability Cloud.

To install the Splunk Python agent in your environment, use the guided setup in the UI or follow the manual installation steps.

Generate customized instructions using the guided setup

The guided setup in the UI generates all the basic installation commands for your environment and application. To use the guided setup:

  1. Log into Splunk Observability Cloud.

  2. Open the Python guided setup. Optionally, you can navigate to the guided setup on your own:

    1. In the navigation menu, select Data Management.

    2. Go to the Available integrations tab, or select Add Integration in the Deployed integrations tab.

    3. In the integration filter menu, select By Product.

    4. Select the APM product.

    5. Select the Python tile to open the Python guided setup.

Install and configure the Splunk Python agent manually

Follow these steps to manually install the Splunk Python agent and specify its minimum configuration.

Note: The examples in this section are for a Linux / Kubernetes environment.

  1. Confirm that you meet the Splunk Python agent requirements.

  2. (For Secure Application users) If you want to use non-default values for these environment variables, set them. The code block shows the default values. For descriptions, see Configure the Python agent for Splunk Observability Cloud:

    BASH 
    export SPLUNK_SECUREAPP_AGENT_ENABLED=true
export SPLUNK_SECUREAPP_DEPENDENCY_INITIAL_DELAY=15
export SPLUNK_SECUREAPP_DEPENDENCY_SCAN_INTERVAL=30
# export SPLUNK_SECUREAPP_LOG_LEVEL=INFO

  3. Install the Secure Application extension (secureapp-python-agent) and the Splunk OpenTelemetry agent/package (splunk-opentelemetry) together:

    BASH 
    #
# Install CSA Extension (secureapp-python-agent) 
#
pip3 install secureapp-python-agent splunk-opentelemetry
    Tip: If you're using a requirements.txt or pyproject.toml file, add splunk-opentelemetry to it.

  4. (For Secure Application users) Set up an OpenTelemetry Collector based on your deployment model. For instructions, see Step 2: Deploy a supported OpenTelemetry Collector

    For example, if you're deploying the Splunk Distribution of OpenTelemetry Collector on Kubernetes with Helm, run a helm install command that includes these flags:

    BASH 
    helm install splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector \ 
--set="splunkObservability.accessToken=<your-access-token>" \ 
--set="splunkObservability.realm=<your-splunk-realm>" \ 
--set="splunkObservability.secureAppEnabled=true" \ 
--set="clusterName=<your-cluster-name>"

  5. Run the opentelemetry-bootstrap tool to scan your current Python environment, detect which libraries you have installed, and automatically install the necessary OpenTelemetry instrumentation packages for them:

    BASH 
    #
# Install the necessary OpenTelemetry instrumentation packages for your libraries  
#      
opentelemetry-bootstrap -a install

    Alternatively, run the bootstrap script to print the supported packages to the console, and add the output to your requirements.txt or pipfile:

    CODE 
    opentelemetry-bootstrap

  6. Send data directly to Splunk Observability Cloud.

    By default, the agent sends all telemetry to the local instance of the Splunk Distribution of OpenTelemetry Collector. To send data directly to Splunk Observability Cloud, set the SPLUNK_ACCESS_TOKEN and SPLUNK_REALM environment variables:

    BASH 
    #
# Send data directly to Splunk Observability Cloud  
#  
export SPLUNK_ACCESS_TOKEN=<access-token>
export SPLUNK_REALM=<realm>

    To obtain a Splunk access token, see Retrieve and manage user API access tokens using Splunk Observability Cloud. To find your Splunk realm, see Configure SSO integrations for Splunk Observability Cloud. For more information on the ingest API endpoints, see Send APM traces.

  7. Configure the Python agent with these environment variables at a minimum.

    For descriptions of these environment variables, see Configure the Python agent for Splunk Observability Cloud:

    • OTEL_SERVICE_NAME
    • OTEL_RESOURCE_ATTRIBUTES
    • OTEL_LOGS_EXPORTER

    Sample minimal configuration for a Kubernetes environment:

    BASH 
    #
# Python agent settings 
#
export OTEL_SERVICE_NAME=<application-name>
export OTEL_RESOURCE_ATTRIBUTES="service.name=<application-name>,deployment.environment=<environment-name>,service.version=<version>"
export OTEL_LOGS_EXPORTER=otlp

  8. (Optional) Add advanced configuration to the Python agent.

    In most cases, the only configuration setting you need to enter is the service name. You can also define other basic settings, like the deployment environment, the service version, and the endpoint, among others.

    For advanced configuration of the Python agent, like changing trace propagation formats, correlating traces and logs, or configuring server trace data, see Configure the Python agent for Splunk Observability Cloud.

  9. (Optional) Add additional instrumentation to your application.

  10. (Optional) If you're instrumenting a Python app that uses Django or uWSGI, see Instrument your Django application.

  11. (For Secure Application users) Integrate this content into your collector configuration file:

    Sample configuration for any OpenTelemetry Collector with Secure Application enabled:

    YAML 
    #
# Add these Secure Application settings into your 
# Splunk Distribution of OpenTelemetry Collector configuration file.
#
# In this setup, applications send standard OTLP telemetry 
# to the collector. The collector routes only Secure Application
# dependency logs to the Secure Application event endpoint
# and adds the backend-specific Secure Application header 
# on that outbound exporter.
#
# Required environment variables:
#   SPLUNK_ACCESS_TOKEN
#   SPLUNK_REALM
#
# Python applications using splunk-opentelemetry[secureapp] should 
# send OTLP logs to this collector. For example:
#   OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://localhost:4318/v1/logs
#
# Do not send general application logs to /v3/event. 
# Keep normal logs on their regular logs pipeline and route only 
# instrumentation_scope.name == "secureapp" to the `logs/secureapp` 
# pipeline.

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318


# ###############################################
# Secure Application filters and exporters START
# ###############################################

processors:
  batch: {}
  filter/drop_secureapp:
    error_mode: ignore
    logs:
      log_record:
        - instrumentation_scope.name == "secureapp"
  filter/keep_secureapp:
    error_mode: ignore
    logs:
      log_record:
        - instrumentation_scope.name != "secureapp"

exporters:
  otlphttp/secureapp:
    logs_endpoint: https://ingest.${SPLUNK_REALM}.observability.splunkcloud.com/v3/event
    headers:
      X-SF-TOKEN: ${SPLUNK_ACCESS_TOKEN}
      X-Splunk-Instrumentation-Library: secureapp

# ###############################################
# Secure Application filters and exporters END
# ###############################################

  # Replace this placeholder with the normal logs exporter for your deployment,
  # for example splunk_hec or otlp/gateway.
  debug/default_logs:
    verbosity: basic

  otlphttp/traces:
    traces_endpoint: https://ingest.${SPLUNK_REALM}.observability.splunkcloud.com/v2/trace/otlp
    headers:
      X-SF-TOKEN: ${SPLUNK_ACCESS_TOKEN}

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlphttp/traces]

    logs/default:
      receivers: [otlp]
      processors: [filter/drop_secureapp, batch]
      exporters: [debug/default_logs]

# ###############################################
# Secure Application pipeline START
# ###############################################

    logs/secureapp:
      receivers: [otlp]
      processors: [filter/keep_secureapp, batch]
      exporters: [otlphttp/secureapp]

# ###############################################
# Secure Application pipeline END
# ###############################################

    Sample configuration for a Helm chart that deploys the collector as a deployment and sets up zero-code instrumentation:

    To deploy the Python agent in Kubernetes, configure the Kubernetes Downward API to expose environment variables to Kubernetes resources.

    The following example shows how to update a deployment to expose environment variables by adding the agent configuration under the .spec.template.spec.containers.env section:

    YAML 
    apiVersion: apps/v1
kind: Deployment
spec:
  selector:
    matchLabels:
      app: your-application
  template:
    spec:
      containers:
        - name: myapp
          env:
            - name: SPLUNK_OTEL_AGENT
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://$(SPLUNK_OTEL_AGENT):4317"
            - name: OTEL_SERVICE_NAME
              value: "<application-name>"
            - name: OTEL_RESOURCE_ATTRIBUTES
              value: "deployment.environment=<environment-name>"

  12. Launch your Python application with the opentelemetry-instrument wrapper.

    Zero-code
    BASH 
    opentelemetry-instrument python3 <application-name>.py
    Code-based

    Code-based (manual) instrumentation isn't supported.

    uWSGI

    To instrument uWSGI applications, see Manually instrument Python applications for Splunk Observability Cloud.

Application metrics are collected by default. See Metrics and attributes collected by the Splunk Distribution of OpenTelemetry Python for more information.

If no data appears in Splunk APM, see Troubleshoot Python instrumentation for Splunk Observability Cloud.