Configure OpenTelemetry Fleet Management

Important: This feature is being released as part of a phased rollout. Access is currently limited to a select group of accounts within specific realms and will be gradually expanded to all accounts across all realms. If you would like to request access to this feature, contact your account representative for further information
OpenTelemetry Fleet Management requires agents to be enrolled so they connect to the centralized Fleet Management Server. By configuring the agents for Fleet Management, each service registers with the Fleet Management Server, reports its identity and health through the OpAMP management path through the Collector.

Depending on the clients (collectors or agents) that you want to manage, you need to:

  • configure OpAMP for Collector

  • configure OpAMP for Agents

Before you begin

You must run a OpenTelemetry Collector on the same host or a reachable network path to:

  • receive agent telemetry over OTLP (data plane)

  • act as the OpAMP HTTP proxy for language agents (management plane).

For a quick overview of the Collector, see Splunk Distribution of the OpenTelemetry Collector.

Configure OpAMP for Collector

In the OpenTelemetry Collector, add the following extension in the agent_config.yaml configuration file:

Note: If you are using OpenTelemetry Collector 0.151.0 or later, apply the splunk.opamp.enabled feature gate to add the extension.
CODE 
extensions:
  opamp/splunk_o11y:
    server:
      http:
        endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
        polling_interval: 30s
        headers:
          X-SF-Token: "<your-token>"

Configure OpAMP for Agents

  1. In the agent_config.yaml configuration file of the OpenTelemetry Collector, add the following extension to act as a proxy for OpAMP requests from agents:

    CODE 
    extensions:
  http_forwarder:
    ingress:
      endpoint: 0.0.0.0:<your-port>
    egress:
      endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
      headers:
        X-SF-Token: "<your-token>"

  2. Configure your agents with the following environment variables:

    Environment variable Type Value Default Description
    SPLUNK_OPAMP_ENABLED boolean true false Enables the OpAMP client in the agent.
    SPLUNK_OPAMP_ENDPOINT string

    http://localhost:<port>/v1/opamp

    		 - Points to the http_forwarder of the collector.

  3. Get the inventory details by using the APIs. See Client Inventory API.

Example of the OpenTelemetry Collector agent_config.yaml file

JSON 
extensions:
  http_forwarder:
    ingress:
      endpoint: 0.0.0.0:<your-port>
    egress:
      endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
      headers:
        X-SF-Token: "<your-token>"

  opamp:
    server:
      http:
        endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp/fm-service
        polling_interval: 30s
        headers:
          X-SF-Token: "<your-token>"

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:<your-port>

exporters:
  debug:

processors:
  batch: {}

service:
  extensions: [http_forwarder, opamp]
  pipelines:
    traces:
      receivers:  [otlp]
      processors: [batch]
      exporters:  [debug]

Parameter Details:

  • ingress.endpoint: the port at which the agent connects to. For example, the Java agent connects to port 4320
  • egress.endpoint: the base URL of the Fleet Management server. (no path. The request path /v1/opamp is forwarded without modifications)
  • egress.headers: the token is added so that the agent does not need to carry it.