Ingestion separation resource spec parameters

Spec parameter reference for ingestion separation custom resources.

Spec parameters for the Queue, IngestorCluster, and ObjectStorage custom resources used for index and ingestion separation.

Queue resource spec parameters

CODE 
apiVersion: enterprise.splunk.com/v4
kind: Queue
metadata:
  name: queue
spec:
  replicas: 3
  provider: sqs
  sqs:
    name: sqs-test
    region: us-west-2
    endpoint: https://sqs.us-west-2.amazonaws.com
    dlq: sqs-dlq-test

The SQS provider is the only supported message queue provider.

Key Type Description
provider string [Required] Provider of message queue (Allowed values: sqs)
sqs SQS [Required if provider=sqs] SQS message queue inputs

SQS message queue inputs:

Key Type Description
name string [Required] Name of the queue
region string [Required] Region where the queue is located
endpoint string [Optional, if not provided formed based on region] AWS SQS Service endpoint
dlq string [Required] Name of the dead letter queue

Changing any of the queue inputs triggers a restart of Splunk so that the appropriate .conf files are refreshed and consumed.

IngestorCluster resource spec parameters

CODE 
apiVersion: enterprise.splunk.com/v4
kind: IngestorCluster
metadata:
  name: ic
spec:
  replicas: 3
  queueRef: 
    name: queue
  objectStorageRef:
    name: os
Note: The queueRef and objectStorageRef fields are required for the IngestorCluster resource. They connect the IngestorCluster to the Queue and ObjectStorage resources.

In addition to the common spec parameters, the IngestorCluster resource provides the following spec configuration parameters:

Key Type Description
replicas integer The number of ingestor peers (minimum of 3, which is the default)

ObjectStorage resource spec parameters

CODE 
apiVersion: enterprise.splunk.com/v4
kind: ObjectStorage
metadata:
  name: os
spec:
  provider: s3
  s3:
    path: ingestion/smartbus-test
    endpoint: https://s3.us-west-2.amazonaws.com

The S3 provider is the only supported object storage provider.

Key Type Description
provider string [Required] Provider of object storage (Allowed values: s3)
s3 S3 [Required if provider=s3] S3 object storage inputs

S3 object storage inputs:

Key Type Description
path string [Required] Remote storage location for messages that are larger than the underlying maximum message size
endpoint string [Optional, if not provided formed based on region] S3-compatible service endpoint

Changing any of the object storage inputs triggers a restart of Splunk so that the appropriate .conf files are refreshed and consumed.