How Dynamic Data Self Storage works

Splunk Cloud Platform moves data to your self storage location when the index meets a configured size or time threshold. Note the following:

  • If an error occurs, a connection issue occurs, or a specified storage bucket is unavailable or full, Splunk Cloud Platform attempts to move the data every 15 minutes until it can successfully move the data.
  • Splunk Cloud Platform does not delete data from the Splunk Cloud Platform environment until it has successfully moved the data to your self storage location.
  • Data is encrypted by SSL during transit to your self storage location. Because Splunk Cloud Platform encryption applies only to data within Splunk buckets, you might want to encrypt data in the target bucket after transit. For Amazon S3 buckets, to ensure your data is protected, enable AES256 SSE-S3 on your target bucket so that data encryption resumes immediately upon arrival at the SSE-S3 bucket. Enabling AES256 SSE-S3 provides server-side encryption with Amazon S3 Managed keys (SSE-S3) only. This feature does not work with KMS keys. For GCP buckets, data encryption is enabled by default.

After Splunk Cloud Platform moves your data to your self storage location, you can maintain the data using your cloud provider's tools. If you need to restore the data so that it is searchable, you can restore the data to a Splunk Enterprise instance. The data is restored to a thawed directory, which exists outside of the thresholds for deletion you have configured on your Splunk Enterprise instance. You can then search the data and delete it when you finish.

Note: When you restore data to a thawed directory on Splunk Enterprise, it does not count against the indexing license volume for the Splunk Enterprise or Splunk Cloud Platform deployment.

The graphic shows a diagram of a Splunk Cloud Platform instance, an Amazon S3 bucket, and a Splunk Enterprise instance. The arrows show how the data moves from Splunk Cloud Platform to the Amazon bucket when the data expires. Then, the arrows show the data moved from the AWS bucket to the Enterprise instance in order to restore the data.