Create an Amazon S3 connection for Edge Processor pipelines

Create an Amazon S3 connection in the Data Management app to allow your pipelines to authenticate and send data to Amazon S3 buckets.

Before you can send data from an Edge Processor to an Amazon S3 bucket, you must first create an Amazon S3 connection in the Data Management app on Splunk Cloud Platform. This connection allows datasets and pipelines to access S3 buckets by authenticating through an IAM role or an access key pair.

  1. In Splunk Cloud Platform, select Data Management from the Apps panel.
  2. Navigate to the Connections page, and then select Create connection.
  3. On the Select data store page, select Amazon S3, then select Next.
  4. On the General page, configure the following options, and then select Next:
    Option name Configuration instructions
    Connection name Enter a unique name for your connection.
    Connection description (Optional) Enter a description for your connection.
    AWS account ID

    Enter the 12-digit ID of the AWS account that has the S3 bucket you want to send data to.

    AWS region Select the region of your AWS account.
    Note: Avoid egress costs by using an AWS account that is in the same region as your Splunk Cloud Platform deployment. You are responsible for any egress charges incurred when sending data from a pipeline to Amazon S3.
  5. On the Select abilities page, do the following:
    1. Select Send data from Edge Processor.
      You can optionally select additional abilities:
      • If you also want to use this connection to send data from Ingest Processor pipelines to S3 buckets, then select Send data from Ingest Processor.

      • If you also want to use this connection to run federated searches over S3 buckets, then select Run federated search.

    2. Select the authentication method you want to use for each ability that you selected during the previous step.
      As a best practice, for the Send data from Edge Processor ability, select Access key. Access key authentication is always supported for this ability, while IAM role authentication is supported only if all the instances of your Edge Processor are installed on Amazon EC2 instances that are associated with the same AWS account as the S3 bucket.
    3. Select Next to continue to the Storage authentication page.

The remaining connection creation steps vary depending on which authentication method you selected for the Send data from Edge Processor ability during step 5. To finish creating this Amazon S3 connection, complete the instructions on the following pages as necessary: