Deploy Splunk POD

Install a Splunk POD cluster from a bastion node using the Splunk Kubernetes Installer.

Deploy a Splunk POD cluster using the Kubernetes Installer for Splunk POD. The installer uses a static cluster configuration file (YAML) as a blueprint for the deployment.

  1. Prepare the system.

    Ensure that all target nodes (controllers and workers) meet the cluster requirements. See Splunk POD Requirements and the Splunk POD CVD.

    • Install and configure the operating system (RHEL 9.6). Disable SELinux and THP, and configure time synchronization (NTP/Chrony).
    • Assign static IP addresses to all nodes and ensure the bastion node has network connectivity to every node. Record these IP addresses for use in the configuration file.
    • Establish secure, non-interactive SSH access from the bastion node using a system user (such as "splunkadmin") with passwordless sudo privileges.
    • Create a DNS A record and TLS certificate with private key. See Configure name-based routing.
  2. Prepare the bastion node.

    Place the following files in a dedicated directory on the bastion server:

    • Installer binary: kubernetes-installer-standalone
    • Splunk Enterprise license: A valid .lic file.
    • SSH private key.

    • TLS certificate/private key.

  3. Create a static cluster configuration file.

    Create a YAML file (for example cluster-config.yaml) on the bastion node. This file is the blueprint for your cluster, specifying the sizing profile, node IP addresses, and apps to install. Add the certificate stanza if you intend to use name-based routing.

    Example configuration for pod-small:

    CODE 
    ---
apiVersion: enterprise.splunk.com/v1beta1
kind: KubernetesCluster
profile: pod-small
licenses: ["/path/to/splunk.lic"]
ssh:
  user: "splunkadmin"
  privateKey: "/path/to/ssh-private-key"
certificate:
  ingress:
    certificate: ./fullchain.pem
    privateKey: ./privkey.pem
controllers:
  - address: "10.10.10.1" # Controller C225
  - address: "10.10.10.2" # Controller C225
  - address: "10.10.10.3" # Controller C225
workers:
  - address: "10.10.10.4" # Search head C225
  - address: "10.10.10.5" # Indexer C245
  - address: "10.10.10.6" # Indexer C245
  - address: "10.10.10.7" # Indexer C245
  - address: "10.10.10.8" # Volume C245
  - address: "10.10.10.9" # Volume C245
  - address: "10.10.10.10" # Volume C245
  - address: "10.10.10.11" # Volume C245
 
clustermanager:
  apps:
    # Apps for indexers
    cluster:
      - "/path/to/indexer-app.tgz"

standalone:
  - name: my-sh
    apps:
      local: ["./path/to/myapp.tgz"]

    Example configuration for pod-medium:

    CODE 
    ---
apiVersion: enterprise.splunk.com/v1beta1
kind: KubernetesCluster
profile: pod-medium
licenses: ["/path/to/splunk.lic", "/path/to/es.lic"]
ssh:
  user: "splunkadmin"
  privateKey: "/path/to/ssh-private-key"
certificate:
  ingress:
    certificate: ./fullchain.pem
    privateKey: ./privkey.pem
controllers:
  - address: "10.10.10.1" # Controller C225
  - address: "10.10.10.2" # Controller C225
  - address: "10.10.10.3" # Controller C225
workers:
  - address: "10.10.10.4" # Search head C225
  - address: "10.10.10.5" # Search head C225
  - address: "10.10.10.6" # Search head C225
  - address: "10.10.10.7" # Indexer C245
  - address: "10.10.10.8" # Indexer C245
  - address: "10.10.10.9" # Indexer C245
  - address: "10.10.10.10" # Indexer C245
  - address: "10.10.10.11" # Volume C245
  - address: "10.10.10.12" # Volume C245
  - address: "10.10.10.13" # Volume C245
  - address: "10.10.10.14" # Volume C245
 
clustermanager:
  apps:
    # Apps for indexers
    cluster:
      - "/path/to/indexer-app.tgz"

searchheadcluster:
  - name: core-shc
    apps:
      cluster:
        - "/path/to/sh-app.tar.gz"

    For field definitions and examples, see the Static cluster configuration file reference.

  4. Install the Kubernetes cluster.

    On the bastion node, run:

    CODE 
    ./kubernetes-installer-standalone -static.cluster cluster-config.yaml -deploy

    Accept the Terms and Conditions during the first run of the installer. Upon acceptance, the installer adds termsConditionsAccepted: true to the cluster configuration file. Remove this key if you share the configuration file.

    Note: Initial installation takes approximately 5–10 minutes. It will take additional time for the cluster to reach a ready state and for app deployment to complete.
Once the installation is complete, proceed to Validate the cluster.

Run pre-flight checks

Verify the integrity of your environment to identify configuration issues early.

The Kubernetes Installer automatically runs a series of pre-flight checks before deploying the cluster to verify the integrity of your environment and catch configuration issues early.

Pre-flight checks include:

  • Static config syntax​

  • Valid SSH key + Splunk license​

  • Valid app paths + scopes​

  • Sufficient hosts for chosen profile​

  • All hosts are reachable​

  • For each host: SSH access w/ sudo​, correct drive mounts​, SELinux disabled​, Chronyd/NTP running​, Correct OS​, and Firewall off

You can manually run pre-flight checks independently of a cluster deployment, as follows:

From the bastion node:

CODE 
./kubernetes-standalone-installer -static.cluster cluster-config.yaml -preflightcheck.only

Validate the cluster

Confirm a successful Splunk POD deployment and access Splunk Enterprise. Allow 10–15 minutes after the initial installation for all cluster components (pods) to reach a fully operational state.

  1. Verify the node status.

    Confirm that all worker nodes have joined the cluster and show Ready status:

    CODE 
    ./kubernetes-installer-standalone -static.cluster cluster-config.yaml -status.workers
  2. Verify the pod status.

    Check the health of individual Kubernetes pods.

    Note: Splunk pods can restart periodically while the system applies configurations.
    CODE 
    ./kubernetes-installer-standalone -static.cluster cluster-config.yaml -status

    Verify that all pods show 1/1 in the READY column and a status of Running.

  3. Access Splunk Web.

    Retrieve your credentials and log in to the user interface. Run the following command to display the admin password and HEC token:

    CODE 
    ./kubernetes-installer-standalone -static.cluster cluster-config.yaml -get.creds

    Navigate to https://<ANY_NODE_IP> and log in to Splunk Enterprise with your admin credentials.

  4. (Optional) Perform advanced debugging.

    Open the kubectl debug shell:

    CODE 
    ./kubernetes-installer-standalone -static.cluster cluster-config.yaml -kubectl

    Verify Splunk tiers by checking the status of the indexer cluster, search head cluster, and cluster manager:

    CODE 
    kubectl get indexerclusters -n splunk -o wide
kubectl get searchheadclusters -n splunk -o wide
kubectl get clustermanagers -n splunk -o wide

Static cluster configuration file reference

Field definitions, app scopes, and configuration examples for the POD YAML file.

Field Value Description
apiVersion enterprise.splunk.com/v1beta1 Schema version of this configuration file.
kind KubernetesCluster Defines the type of resource being created.
profile pod-small, pod-medium, pod-large, pod-xlarge POD sizing option.
licenses List of file paths Absolute path to your Splunk Enterprise license file and your ES or ITSI license file on the bastion node.
ssh user and privateKey The sudo-enabled user and private key path used to access all nodes.
certificate TLS certificate and private key Configuration for name-based routing using a TLS certificate aligning with customer-managed DNS configurations.
controllers List of 3 IP addresses Includes the three control plane servers.
workers List of 4+ IP addresses Includes all search heads, indexers, and volume servers.
clustermanager Nested apps block for the indexer cluster Configuration for indexer cluster including the cluster manager and indexers.
licensemanager Nested apps block for the license manager. Configuration for license manager.
searchheadcluster Array of named search head clusters Configuration for search head clusters. Applies to pod-medium, pod-large, and pod-xlarge options only.
standalone Array of named instances Configuration for standalone search heads. Applies to pod-small option only.

About app scopes

The scope determines how the installer distributes an app across the Kubernetes cluster:

cluster
The installer deploys the app through the cluster manager (for indexers) or the Search Head Cluster Deployer (for search heads). Use this scope to distribute apps across the indexer tier or search head clusters, including ITSI, if applicable.
local
The installer deploys the app only on the selected instance type. Use this scope for standalone search heads to deploy non-ES apps, including ITSI.
premium
Use this scope to deploy the Splunk Enterprise Security (ES) premium app only. The premium scope is supported in standalone and searchheadcluster stanzas.

Configure standalone search heads for pod-small

The pod-small option supports up to two standalone search heads. To enable UI access for this profile, you must configure at least one standalone search head.

When using either ES or ITSI, you can deploy two standalone search heads: the first as an independent search head and the second as a host for either Enterprise Security or ITSI.

To configure these instances, specify the search heads as an array in the standalone stanza of the cluster configuration file, assigning each a unique name.

Note: Search head tiers cannot be renamed after deployment.

You can access the first search head UI on port 8000 and the second on port 8001 using any worker node IP address. If you are configuring name-based routing, you can also access standalone search heads using https://<search head name>.<your domain>.

CODE 
standalone: 
- name: es-sh
  apps: 
    local: 
      - "/path/to/myapp.spl"
     premium: 
      - "/path/to/splunk-es.spl"
- name: my-sh
  apps: 
    local: 
      - "/path/to/myapp.spl"

Configure SHCs for pod-medium, pod-large, and pod-xlarge

The pod-medium, pod-large, and pod-xlarge options each support up to two search head clusters (SHC). To enable UI access, you must configure at least one SHC.

When using either ES or ITSI, you can deploy two clusters: the first as a standard SHC, and the second for either Enterprise Security or ITSI. Each cluster is fixed at three search heads.

To configure these instances, specify the SHCs as an array in the searchheadcluster stanza of the cluster configuration file, assigning each a unique name.

Note: Search head tiers cannot be renamed after deployment.

You can access the first SHC on port 443 and the second on port 8100 using any worker node IP address. If you are configuring name-based routing, you can also access search head clusters using https://<search head cluster name>.<your domain>.

CODE 
searchheadcluster: 
- name: core-shc
  apps: 
    cluster: ["./performance-insights-for-splunk_118.tgz"]
- name: es-shc
  apps: 
    cluster: []
    premium: ["./splunk_app_es-8.1.1-176740.tgz"]