Splunk Enterprise receiver

The Splunk Enterprise receiver enables you to ingest performance metrics describing the operational status of your Splunk Enterprise deployment.

The Splunk Enterprise receiver is a pull based tool which enables you to ingest performance metrics describing the operational status of your Splunk Enterprise deployment to an appropriate observability tool. The receiver uses several different data sources, including the introspection API endpoint, to gather these metrics and serializes results from ad-hoc searches. For more information, see the REST API Reference Manual in Splunk documentation.

The supported pipeline type is metrics. See Process your data with pipelines for more information.

CAUTION: This receiver targets those responsible for the maintenance and care of a Splunk Enterprise deployment, and aims to use OpenTelemetry and observability toolsets. Be careful when enabling the receiver, since running searches can effect your Splunk Enterprise deployment and introspection might fail to report for Splunk Cloud Platform deployments.

Get started

Follow these steps to configure and activate the component:

Deploy the Splunk Distribution of the OpenTelemetry Collector to your host or container platform:
Configure the Splunk Enterprise receiver as described in the next section.
Restart the Collector.

Sample configuration

To activate the Splunk Enterprise receiver, add splunkenterprise to the receivers section of your configuration file:

receivers:
  splunkenterprise:

To complete the configuration, include the receiver in the metrics pipeline of the service section of your configuration file:

service:
  pipelines:
    metrics:
      receivers: [splunkenterprise]

Configuration options

The following settings are required:

basicauth. A configured stanza for the basicauthextension. Learn more at Basicauth extension.
auth. No default. String name referencing your auth extension.
endpoint. No default. Your Splunk Enterprise hos endpoint.

The following settings are optional:

collection_interval. 10m by default. The time between scrape attempts.
timeout. 60s by default. The time the scrape function waits for a response before returning an empty value.

The following applies to the Splunk Enterprise receiver configuration:

Omitting any of the mandatory settings might cause your receiver to fail to compile or result in 4/5xx return codes during scraping.
Set these parameters for each Splunk instance type (indexer, search head, or cluster master) from which you wish to pull metrics from.
- Currently only 1 instance type is accepted per configured receiver instance. If you have 3 different "indexer" type instances, to pull metrics you need to configure 3 different splunkenterprise receivers, 1 for each indexer node.

Configuration example

See the following configuration example for the Splunk Enterprise receiver:

extensions:
  basicauth/indexer:
      client_auth:
          username: admin
          password: securityFirst
  basicauth/cluster_master:
      client_auth:
          username: admin
          password: securityFirst

receivers:
  splunkenterprise:
      indexer:
          auth:
            authenticator: basicauth/indexer
          endpoint: "https://localhost:8089"
          timeout: 45s
      cluster_master:
          auth:
            authenticator: basicauth/cluster_master
          endpoint: "https://localhost:8089"
          timeout: 45s

exporters:
  logging:
    loglevel: info

service:
  extensions: [basicauth/indexer, basicauth/cluster_master]
  pipelines:
    metrics:
      receivers: [splunkenterprise]
      exporters: [logging]

Metrics

The following metrics, resource attributes, and attributes are available.

Note: The SignalFx exporter excludes some available metrics by default. Learn more about default metric filters in List of metrics excluded by default.

included

https://raw.githubusercontent.com/splunk/collector-config-tools/main/metric-metadata/splunkenterprisereceiver.yaml

Activate or deactivate specific metrics

You can activate or deactivate specific metrics by setting the enabled field in the metrics section for each metric. For example:

receivers:
  samplereceiver:
    metrics:
      metric-one:
        enabled: true
      metric-two:
        enabled: false

The following is an example of host metrics receiver configuration with activated metrics:

receivers:
  hostmetrics:
    scrapers:
      process:
        metrics:
          process.cpu.utilization:
            enabled: true

Note: Deactivated metrics aren’t sent to Splunk Observability Cloud.

Billing

If you’re in a MTS-based subscription, all metrics count towards metrics usage.
If you’re in a host-based plan, metrics listed as active (Active: Yes) on this document are considered default and are included free of charge.

Learn more at Infrastructure Monitoring subscription usage (Host and metric plans).

Troubleshooting

If you are a Splunk Observability Cloud customer and are not able to see your data in Splunk Observability Cloud, you can get help in the following ways.

Available to Splunk Observability Cloud customers

Submit a case in the Splunk Support Portal.
Contact Splunk Support.

Available to prospective customers and free trial users

Ask a question and get answers through community support at Splunk Answers.
Join the Splunk #observability user group Slack channel to communicate with customers, partners, and Splunk employees worldwide. To join, see Chat groups.

Documentation