Set up OpenTelemetry Fleet Management for Inventory management

Learn how to configure OpAMP for collectors and agents.

OpenTelemetry Fleet Management requires agents to be enrolled so they connect to the centralized Fleet Management Server. By configuring the agents for Fleet Management, each service registers with the Fleet Management Server, reports its identity and health through the OpAMP management path through the Collector.

Depending on the clients (collectors or agents) that you want to manage, you need to:

  • configure OpAMP for Collector

  • configure OpAMP for Agents

Before you begin

You must run a OpenTelemetry Collector on the same host or a reachable network path to:

  • receive agent telemetry over OTLP (data plane)

  • act as the OpAMP HTTP proxy for language agents (management plane).

For a quick overview of the Collector, see Splunk Distribution of the OpenTelemetry Collector.

Configure OpAMP for collector

  1. In the OpenTelemetry Collector, you need the following extensions in the agent_config.yaml configuration file:

    Note:

    • If you are using OpenTelemetry Collector version prior to 0.151.0, add the extensions manually.

    • If you are using OpenTelemetry Collector version 0.151.0 to 0.157, apply the splunk.opamp.enabled feature gate to add the extensions.

    • If you are using OpenTelemetry Collector 0.158.0 or later, the extensions are enabled by default.

    CODE 
    extensions:
  opamp/splunk_o11y:
	agent_description:
      include_resource_attributes: true
    server:
      http:
        endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
        polling_interval: 30s
        headers:
          X-SF-Token: "<your-token>"
  2. (Optional) Set the deployment environment in the Collector internal telemetry resource attributes.This setting populates the Environment field in the Fleet management UI and associates OpAMP data with the same environment as your APM data. In the Collector configuration file, add deployment.environment.name under service::telemetry::resource::attributes.

    For example, if you use the default agent Collector configuration on Linux at /etc/otel/collector/agent_config.yaml, enable this by following the comments for deployment.environment.name as mentioned in the following configuration snippet:

    CODE 
    service:
  telemetry:
    resource:
      attributes:
        - name: otelcol.service.mode
          value: agent
        # Optional: Add a deployment environment to the Collector's internal telemetry.
        # To use the deprecated attribute, change the key to "deployment.environment".
        - name: deployment.environment.name
          value: staging

  3. If you changed the Collector configuration in the previous step, restart the Collector service.

Configure OpAMP for agents

  1. In the agent_config.yaml configuration file of the OpenTelemetry Collector, add the following extension to act as a proxy for OpAMP requests from agents:

    CODE 
    extensions:
  http_forwarder:
    ingress:
      endpoint: 0.0.0.0:<your-port>
    egress:
      endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
      headers:
        X-SF-Token: "<your-token>"

  2. Configure your agents with the following environment variables:

    Environment variable Type Value Default Description
    SPLUNK_OPAMP_ENABLED boolean true false Enables the OpAMP client in the agent.
    SPLUNK_OPAMP_ENDPOINT string

    http://localhost:<port>/v1/opamp

    		 - Points to the http_forwarder of the collector.

  3. Get the inventory details by using the APIs. See Client Inventory API.

Example of the OpenTelemetry collector agent_config.yaml file

JSON 
extensions:
  http_forwarder:
    ingress:
      endpoint: 0.0.0.0:<your-port>
    egress:
      endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp
      headers:
        X-SF-Token: "<your-token>"

  opamp:
    agent_description:
      include_resource_attributes: true 
    server:
      http:
        endpoint: https://ingest.<realm>.observability.splunkcloud.com/v1/opamp/fm-service
        polling_interval: 30s
        headers:
          X-SF-Token: "<your-token>"

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:<your-port>

exporters:
  debug:

processors:
  batch: {}

service:
  extensions: [http_forwarder, opamp]
  pipelines:
    traces:
      receivers:  [otlp]
      processors: [batch]
      exporters:  [debug]

Parameter Details:

  • ingress.endpoint: the port at which the agent connects to. For example, the Java agent connects to port 4320
  • egress.endpoint: the base URL of the Fleet Management server. (The request path /v1/opamp is forwarded without modifications)
  • egress.headers: the token is added so that the agent does not need to carry it.