Example: Deploy physical separation with SOK

Example sequence for deploying a physically separated ingestion and indexing topology with SOK.

Before you begin, review the prerequisites in Configure physical separation of indexing and ingestion with SOK.

The following is the example sequence of installing apps to deploy a new physical separation topology. For details for each step, see Configure physical separation of indexing and ingestion with SOK.

Note: This example uses the workload identity method to configure access to the SQS queue and S3 bucket. This method requires configuring a service account.
  1. Install Splunk Operator for Kubernetes (SOK).
  2. Configure a service account.
  3. Create the Queue custom resource.

    Create the external message queue and dead-letter queue. Then apply the Queue YAML manifest that references them.

  4. Create the ObjectStorage custom resource.

    Create the S3 bucket and grant the required access to the configured identity or credentials. Then apply the ObjectStorage YAML manifest that references the object-storage location.

    The ObjectStorage resource does not create or manage the external bucket.

  5. Create the IngestorCluster custom resource.

    Apply the IngestorCluster YAML manifest. Configure it to reference the Queue and ObjectStorage resources.

    The IngestorCluster receives and processes incoming data and publishes the processed data through SmartBus.

  6. Integrate the separate indexer cluster.

    Configure the separate indexer cluster to retrieve processed data through the same SmartBus queue and object-storage location referenced by the IngestorCluster.

    The separate indexer cluster is not managed by SOK.

  7. Optionally configure a Kubernetes HorizontalPodAutoscaler (HPA).

    Configure the HPA to adjust the number of IngestorCluster replicas according to ingestion demand. See Scaling the ingestion tier with HPA.

    Note:

    The HPA is managed separately by the customer.