Remote configuration for AlwaysOn Profiling

Using environment variables

You can enable OpAMP remote configuration for the Java agent with the following environment variables:

If required, configure the agent’s initial profiling settings using the environment variables or system properties. See Java settings for AlwaysOn Profiling.

Using file-based configuration

You can enable OpAMP remote configuration in the agent’s declarative configuration file. Set OTEL_CONFIG_FILE to the path of that file:

OTEL_CONFIG_FILE=/path/to/agent-config.yaml

Add the following configuration to the startup YAML file:

distribution:
  splunk:
    opamp/development:
      endpoint: <URL of the OpAMP server>
      polling_interval: 30000
      features:
        remote_config:

Configure the initial profiling settings of the agent in the startup YAML. See File-based configuration for Java in Splunk Observability Cloud.

Sample payload

After you enable OpAMP remote configuration, the OpAMP server sends profiling settings to the agent. The remote payload uses the splunk.remote.config key and application/yaml content type. This is a server-sent payload, separate from the agent’s environment variables and system properties. For more information, see Open Agent Management Protocol specification.

For example, this payload enables CPU profiling with a sampling interval of 1001 milliseconds and memory profiling:

distribution:
  splunk:
    profiling:
      always_on:
        cpu_profiler:
          sampling_interval: 1001
        memory_profiler:

This payload enables snapshot profiling:

distribution:
  splunk:
    profiling:
      callgraphs:
        sampling_interval: 10
        selection_probability: 0.01

Including cpu_profiler, memory_profiler, or callgraphs enables the corresponding profiler. Omitting one of these sections from an accepted remote configuration stops that profiler. If you omit a setting from a profiler section, the agent uses its default value. If the remote payload does not contain the distribution.splunk.profiling node, the agent skips profiler-related remote configuration processing.

The selection_probability value must be greater than 0 and no greater than 1. After the agent accepts a remote configuration, that configuration takes precedence over startup configuration for the profiling settings it contains.

Using environment variables

You can enable OpAMP remote configuration for the Node.js agent with the following environment variables:

If required, configure the initial profiling settings of the agent by using the environment variables or system properties. See Node.js settings for AlwaysOn Profiling.

Using file-based configuration

Docs URL:

You can enable OpAMP remote configuration in the agent’s declarative configuration file. Set OTEL_CONFIG_FILE to the path of that file:

OTEL_CONFIG_FILE=/path/to/agent-config.yaml

Add the following configuration to the startup YAML file:

distribution:
  splunk:
    opamp/development:
      endpoint: <URL of the OpAMP server>
      polling_interval: 30000
      features:
        remote_config:

Configure the agent’s initial profiling settings in the startup YAML. See File-based configuration for Node.js in Splunk Observability Cloud.

Sample payload

After you enable OpAMP remote configuration, the OpAMP server sends profiling settings to the agent. The remote payload uses the splunk.remote.config key and application/yaml content type. This is a server-sent payload, separate from the agent’s environment variables and system properties. For more information, see Open Agent Management Protocol specification.

For example, this payload enables CPU profiling with a sampling interval of 1001 milliseconds and memory profiling:

distribution:
  splunk:
    profiling:
      always_on:
        cpu_profiler:
          sampling_interval: 1001
        memory_profiler:

This payload enables snapshot profiling:

distribution:
  splunk:
    profiling:
      callgraphs:
        sampling_interval: 10
        selection_probability: 0.01

Including cpu_profiler, memory_profiler, or callgraphs enables the corresponding profiler. Omitting one of these sections from an accepted remote configuration stops that profiler. If you omit a setting from a profiler section, the agent uses its default value. If the remote payload does not contain the distribution.splunk.profiling node, the agent skips profiler-related remote configuration processing.

The selection_probability value must be greater than 0 and no greater than 1. After the agent accepts a remote configuration, that configuration takes precedence over startup configuration for the profiling settings it contains.

Supported languages

The following languages support remote configuration for AlwaysOn Profiling

Supported variables for remote configuration for AlwaysOn Profiling

The following settings support remote configuration for AlwaysOn Profiling:

Permissions

The following role-based permissions apply to users:

For information about the OpenAPI specifications, see APM Profiling Policy API.

The table provides details about the fields that you need to set:

Manage profiling operations

• Select Start profiling to submit the configuration and create or update the profiling policy. After submission, the page displays the active policy state, target information, configured settings, remaining time or expiry, and actions such as edit or stop profiling data.

• Select Edit to reopen the active policy and change the instances, profiling mode, sampling interval, or duration. You can't change the environment.

• Select Stop profiling to cancel the profiling settings.

Add multiple environments

Select +Add environment to add and configure multiple environments independently in the same workflow.

Select the Code profiling and Memory profiling tabs to view the relevant code profiling and memory profiling details with service and environment context.