Deploy a multipart indexer cluster

Deploy a multipart indexer cluster across availability zones.

Connect multiple IndexerCluster resources to a ClusterManager resource, each constrained to run within a dedicated zone and configured with a hardcoded site.

Advantages of a multipart IndexerCluster:

  • The operator performs some operations per site, which mitigates the risk of impact on the whole cluster (for example, Splunk upgrades and scaling up resources).
  • Specific indexer services are created per site, allowing you to send events to indexers located in the same zone and avoiding the possible cost of cross-zone traffic. Indexer discovery from the cluster manager can do this for forwarders, but this solution also covers HTTP/HEC traffic.
Note: All IndexerCluster resources must be located in the same namespace.
  1. Deploy the cluster manager.

    The following example defines the image version in the resource to control the upgrade cycle. For all available default parameters, see default.yml.spec.md in the splunk-ansible repository.

    CODE 
    cat <<EOF | kubectl apply -n splunk-operator -f -
---
apiVersion: enterprise.splunk.com/v4
kind: ClusterManager
metadata:
  name: example
  finalizers:
  - enterprise.splunk.com/delete-pvc
spec:
  defaults: |-
    splunk:
      site: site1
      multisite_master: localhost
      all_sites: site1,site2,site3
      multisite_replication_factor_origin: 1
      multisite_replication_factor_total: 2
      multisite_search_factor_origin: 1
      multisite_search_factor_total: 2
      idxc:
        search_factor: 2
        replication_factor: 2
      apps_location:
        - "https://example.com/splunk-apps/app3.tgz"
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: failure-domain.beta.kubernetes.io/zone
            operator: In
            values:
            - zone-1a
EOF
  2. Deploy the indexer sites.

    Create an IndexerCluster CR for each required site with zone affinity specified as needed:

    CODE 
    cat <<EOF | kubectl apply -n splunk-operator -f -
---
apiVersion: enterprise.splunk.com/v4
kind: IndexerCluster
metadata:
  name: example-site1
  finalizers:
  - enterprise.splunk.com/delete-pvc
spec:
  replicas: 2
  clusterManagerRef:
    name: example
  defaults: |-
    splunk:
      multisite_master: splunk-example-cluster-manager-service
      site: site1
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: failure-domain.beta.kubernetes.io/zone
            operator: In
            values:
            - zone-1a
EOF
    Note:
    • The value of the zone label (for example, zone-1a for label failure-domain.beta.kubernetes.io/zone) is specific to each cloud provider and should be changed based on the cloud provider you are using.
    • Starting in Kubernetes v1.17, the label failure-domain.beta.kubernetes.io/zone is deprecated in favor of topology.kubernetes.io/zone. See the official documentation.