Elasticsearch receiver

The Elasticsearch receiver queries the Elasticsearch node stats, cluster health and index stats endpoints in order to scrape metrics from a running Elasticsearch cluster.

The Elasticsearch receiver queries Elasticsearch’s node stats, cluster health and index stats endpoints to scrape metrics from a running Elasticsearch cluster. The supported pipeline type is metrics. See Process your data with pipelines for more information.

Note: Out-of-the-box dashboards and navigators aren’t supported for the Elasticsearch receiver.

To learn more about the queried endpoints, see the following topics in the Elastic documentation:

Prerequisites

This receiver supports Elasticsearch versions 7.9 or higher.

If Elasticsearch security features are enabled, you must have either the monitor or manage cluster privilege. See https://www.elastic.co/guide/en/elasticsearch/reference/current/authorization.html for role-based access control and https://www.elastic.co/guide/en/elasticsearch/reference/current/security-privileges.html for security privileges in the Elastic documentation for more information.

Deploy the collector

See Deploy the Splunk Distribution of the OpenTelemetry Collector.

Configure the receiver

Edit your OpenTelemetry Collector configuration file as follows.

  1. Add elasticsearch to the receivers section of your configuration file:

    CODE 
    receivers:
  elasticsearch:
    metrics:
      elasticsearch.node.fs.disk.available:
        enabled: false
    nodes: ["_local"]
    skip_cluster_metrics: true
    indices: [".geoip_databases"]
    endpoint: http://localhost:9200
    username: otel
    password: password
    collection_interval: 10s

  2. Include the receiver in the metrics pipeline of the service section of your configuration file:

    CODE 
    service:
  pipelines:
    metrics:
      receivers:
        - elasticsearch

Advanced configurations

Optional settings

The following settings are optional:

  • nodes. ["_all"] by default. Allows you to specify node filters that define which nodes are scraped for node-level and cluster-level metrics.

  • skip_cluster_metrics. false by default. If true, cluster-level metrics are not scraped.

  • indices. ["_all"] by default. Allows you to specify index filters that define which indices are scraped for index-level metrics.

  • endpoint. http://localhost:9200 by default. The base URL of the Elasticsearch API for the cluster to monitor.

  • username. No default. Specifies the username used to authenticate with Elasticsearch using basic auth.

  • password. No default. Specifies the password used to authenticate with Elasticsearch using basic auth.

  • collection_interval. 10s by default. This receiver collects metrics on an interval determined by this setting. This value must be a string readable by Golang’s time.ParseDuration. See https://pkg.go.dev/time#ParseDuration.

    • On larger clusters, you might need to increase this interval, as querying Elasticsearch for metrics takes longer on clusters with more nodes.

  • initial_delay. 1s by default. Defines how long this receiver waits before starting.

Activate or deactivate specific metrics

You can activate or deactivate specific metrics by setting the enabled field in the metrics section for each metric. For example:

YAML 
receivers:
  samplereceiver:
    metrics:
      metric-one:
        enabled: true
      metric-two:
        enabled: false

The following is an example of host metrics receiver configuration with activated metrics:

YAML 
receivers:
  hostmetrics:
    scrapers:
      process:
        metrics:
          process.cpu.utilization:
            enabled: true
Note: Deactivated metrics aren’t sent to Splunk Observability Cloud.
Billing

  • If you’re in a MTS-based subscription, all metrics count towards metrics usage.

  • If you’re in a host-based plan, metrics listed as active (Active: Yes) on this document are considered default and are included free of charge.

Learn more at Infrastructure Monitoring subscription usage (Host and metric plans).

Restart the collector

The restart command varies depending on what platform you deployed the collector on and what tool you used to deploy it. Here are general examples of the restart command:

Linux

Linux with installer script:

BASH 
sudo systemctl restart splunk-otel-collector
Windows

Windows with installer script:

BASH 
stop-service splunk-otel-collector
start-service splunk-otel-collector
Kubernetes

Kubernetes with Helm:

BASH 
helm upgrade your-splunk-otel-collector splunk-otel-collector-chart/splunk-otel-collector -f your-override-values.yaml

where splunk-otel-collector-chart is the name you gave to the Helm chart in the helm repo add command.

Settings reference

The following table shows the configuration options for the Elasticsearch receiver:

included

https://raw.githubusercontent.com/splunk/collector-config-tools/main/cfg-metadata/receiver/elasticsearch.yaml

Metrics reference

The following metrics, resource attributes, and attributes, are available.

Note:

Metric availability:

The following metrics are available with versions:

  • elasticsearch.indexing_pressure.memory.limit. Available in versions 7.10 or higher.

  • elasticsearch.node.shards.data_set.size. Available in versions 7.13 or higher.

  • elasticsearch.cluster.state_update.count. Available in versions 7.16.0 or higher.

  • elasticsearch.cluster.state_update.time. Available in versions 7.16.0 or higher.

included

https://raw.githubusercontent.com/splunk/collector-config-tools/main/metric-metadata/elasticsearchreceiver.yaml

Troubleshooting

See Troubleshoot the Splunk OpenTelemetry Collector.