Configure self storage in Amazon S3

To configure a new self storage location in Amazon S3, you must create an S3 bucket in your AWS environment, and configure the S3 bucket as a new self storage location in the Splunk Cloud Platform UI. When you configure the S3 bucket as a new storage location, Splunk Cloud Platform generates a resource-based bucket policy that you must copy/paste to your S3 bucket to grant Splunk Cloud the required access permissions.

For information on how to create and manage Amazon S3 buckets, see the AWS documentation. For information on the differences between AWS identity-based policies and resource-based policies, see https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_identity-vs-resource.html

Create an Amazon S3 bucket in your AWS environment

When creating an Amazon S3 bucket, follow these important configuration guidelines:

  • Region: You must provision your Amazon S3 bucket in the same region as your Splunk Cloud Platform environment.
  • Naming: When you name the S3 bucket, it must include the Splunk prefix provided to you and displayed in the UI under the AWS S3 bucket name field. Enter the prefix before the rest of the bucket name. This prefix contains your organization's Splunk Cloud ID, which is the first part of your organization's Splunk Cloud URL, and a 12-character string. The complete S3 bucket name has the following syntax:

Splunk Cloud ID-{12-character string}-{your bucket name}

For example, if you administer Splunk Cloud Platform for Buttercup Cloudworks, and your organization's Splunk Cloud URL is buttercupcloudworks.splunkcloud.com, then your Splunk Cloud ID is buttercupcloudworks. The image shows the following example prefix you'd see when configuring an S3 bucket using the New Self Storage Location dialog box:

buttercupcloudworks-rs73hfjie674-{your bucket name}

This example shows the Splunk prefix needed when naming a new AWS S3 bucket on the New Self Storage Location dialog box.

If you do not use the correct prefix, Splunk cannot write to your bucket. By default, your Splunk Cloud Platform instance has a security policy applied which disallows write operations to S3 buckets that do not include your Splunk Cloud ID. This security policy allows the write operation only for those S3 buckets that you create for the purpose of storing your expired Splunk Cloud data.

Note: If you have enabled AES256 SSE-S3 on your target bucket, the data will resume encryption at rest upon arrival at the SSE-S3 bucket.

Configure a self storage location for the Amazon S3 bucket

To configure your Amazon S3 bucket as a self storage location in Splunk Cloud Platform:

  1. In Splunk Web, click Settings > Indexes > New Index.
  2. In the Dynamic Data Storage field, click the radio button for Self Storage.
  3. Click Create a self storage location.
    The Dynamic Data Self Storage page opens.
  4. Give your location a Title and an optional Description.
  5. In the Amazon S3 bucket name field, enter the name of the S3 bucket that you created.
  6. (Optional) Enter the bucket folder name.
  7. Click Generate. Splunk Cloud Platform generates a bucket policy.
  8. Copy the bucket policy to your clipboard. Note: Customers with an SSE-S3 encrypted bucket must use the default policy and not modify the policy in any way.
  9. In a separate window, navigate to your AWS Management console and apply this policy to the S3 bucket you created earlier.
  10. In the Self Storage Locations dialog, click Test.
    Splunk Cloud writes a 0 KB test file to the root of your S3 bucket to verify that Splunk Cloud Platform has permissions to write to the bucket. A success message displays, and the Submit button is enabled.
  11. Click Submit.
  12. In the AWS Management Console, verify that the 0 KB test file appears in the root of your bucket.
Note: You cannot edit or delete a self storage location after it is defined, so verify the name and description before you save it.