Manage DDSS self storage locations

The Admin Config Service (ACS) API lets you manage DDSS self storage locations for your Splunk Cloud Platform indexes. You can use the ACS API to view and configure self storage locations on AWS, GCP, and Azure programmatically without using the Splunk Web UI.

This topic covers how to manage DDSS self storage locations only. For instructions on how to configure indexes for use with DDSS/DDAA, see Manage indexes in Splunk Cloud Platform.

For more information on how to configure DDSS self storage locations, see Store expired Splunk Cloud Platform data in your private archive in the Splunk Cloud Platform Admin Manual.

Requirements

To manage DDSS self storage locations using the ACS API:

  • You must have Splunk Cloud Platform version 8.2.2106 or higher. For managing self storage on Azure, you must have Splunk Cloud Platform version 10.4.2604 or higher.
  • You must have the sc_admin role.
  • Your deployment must have one or more separate search heads or a search head cluster. ACS is not supported on single instance deployments.
  • You must have an existing S3 or GCS bucket pre-created in the same AWS or GCP region as your Splunk Cloud Platform deployment.

Set up the ACS API

Before using the ACS API, you must download the ACS Open API 3.0 specification, which includes the parameters, response codes, and other data you need to work with the ACS API. You must also create an authentication token in Splunk Cloud Platform for use with ACS endpoint requests. For details on how to set up the ACS API for index management, see Set up the ACS API.

Manage DDSS self storage locations using the ACS API

You can use the ACS API to list, describe, and configure DDSS self storage locations.

Before you can configure a self storage location, you must create an S3 bucket, GCS bucket, or Azure container in the same AWS, GCP, or Azure region as your Splunk Cloud Platform deployment.

Note: ACS does not support modifying and deleting self storage locations.

List self storage locations

To view a list of all valid self storage locations configured for your deployment, send an HTTP GET request to the cloud-resources/self-storage-locations/buckets endpoint. For example:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/buckets' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

The request returns information about each self storage location, including bucketName, bucketPath, title and uri. For example:

PYTHON 
{
    "selfStorageLocations": [
        {
            "bucketName": "<bucket-name>",
            "bucketPath": "<bucket-name>/dup-title-ui",
            "description": "Test duplicate title from UI",
            "folder": "dup-title-ui",
            "title": "test-bucket-1-with-message",
            "uri": "s3://<bucket-name>/dup-title-ui"
        },
        {
            "bucketName": "<bucket-name>",
            "bucketPath": "<bucket-name>/some-folder",
            "description": "Test",
            "folder": "some-folder",
            "title": "test-bucket-1-with-message",
            "uri": "s3://<bucket-name>/some-folder"
        },
        {
            "bucketName": "<bucket-name>",
            "bucketPath": "<bucket-name>/with-message",
            "description": "Test configuring ddss with ACS and show async message",
            "folder": "with-message",
            "title": "test-bucket-1-with-message",
            "uri": "s3://<bucket-name>/with-message"
        }
    ]
}

Describe self storage locations

To view information about a specific self storage location, send an HTTP GET request to the cloud-resources/self-storage-locations/buckets/{bucketPath} endpoint, specifying the bucketPath value in the request URL, where bucketPath is a unique identifier that combines <bucket-name>/<bucket-folder>. The bucket path must be URL encoded. For example, <bucket-name> must be passed as <bucket-name>%2Fsome-folder, as shown:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/buckets/<bucket-name>%2Fsome-folder' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

The response shows information about the specific self storage location. For example:

For AWS deployments:

JSON 
{
    "bucketName": "<bucket-name>",
    "bucketPath": "<bucket-name>/some-folder",
    "description": "Test",
    "folder": "some-folder",
    "title": "test-bucket-1-with-message",
    "uri": "s3://<bucket-name>/some-folder"
}

For GCP deployments:

JSON 
{
    "bucketName": "<bucket-name>",
    "bucketPath": "<bucket-name>/some-folder",
    "description": "Test",
    "folder": "some-folder",
    "title": "test-bucket-with-message",
    "uri": "gs://<bucket-name>/some-folder"
}

For Azure deployments:

JSON 
{
    "bucketName": "<bucket-name>",
    "bucketPath": "<bucket-name>/some-folder",
    "description": "Test",
    "folder": "some-folder",
    "title": "test-bucket-with-message",
    "uri": "az://<bucket-name>/some-folder"
}
Note: The bucketPath value combines bucketName and folder and is used as an identifier for the self storage location.

Get prefix to configure a bucket (AWS and GCP)

To configure a new self storage location on AWS or GCP, you must have the predefined bucket name prefix provided by Splunk Cloud Platform. This prefix contains your organization's Splunk Cloud ID, which is the first part of your Splunk Cloud URL, and a 12-character string (for AWS) or 4-character string (for GCP). For example, an AWS S3 bucket name has the following syntax:

Splunk Cloud ID-{12-character string}-{your bucket name}

Note:

To configure a new self storage location on Azure, you must provide the storage account resource ID instead of a bucket name prefix. Requesting a bucket prefix is not applicable and returns a 400 Bad Request status code.

To retrieve the prefix for your AWS or GCP bucket, send an HTTP GET request to the cloud-resources/self-storage-locations/configs/prefix endpoint. For example:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/configs/prefix' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

The response includes the prefix. For example:

JSON 
{
    "message": "Please create a bucket in the same region as your Splunk Cloud environment. The bucket must have 'acs-play-noah-aws-iycf10l9z5nl-' as the prefix in the name",
    "prefix": "acs-play-noah-aws-iycf10l9z5nl-"
}

For more information on bucket name prefixes, see Configure self storage locations in the Splunk Cloud Platform Admin Manual.

Get IAM policy for AWS S3 bucket

When you configure a new self storage location for AWS, you must specify an S3 bucket policy. This policy allows your Splunk Cloud Platform deployment to access your AWS S3 bucket.

To generate an AWS S3 bucket policy, send an HTTP GET request to the cloud-resources/self-storage-locations/buckets/{bucketName}/policy endpoint, specifying the AWS S3 bucket name in the URL. For example:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/buckets/<bucket-name>/policy' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

The response includes the AWS S3 bucket policy, which you must apply to your S3 bucket in AWS. For example:

JSON 
{
    "message": "Please copy and apply this bucket policy to your S3 bucket in AWS. Please refer to https://docs.splunk.com/Documentation/SplunkCloud/latest/Admin/DataSelfStorage for more info.",
    "policy": {
        "Statement": [
            {
                "Action": [
                    "s3:PutObject",
                    "s3:ListBucket"
                ],
                "Effect": "Allow",
                "Principal": {
                    "AWS": "arn:aws:iam::594195655983:role/acs-play-noah-aws"
                },
                "Resource": [
                    "arn:aws:s3:::acs-play-noah-aws-iycf10l9z5nl-some-bucket",
                    "arn:aws:s3:::acs-play-noah-aws-iycf10l9z5nl-some-bucket/*"
                ]
            }
        ],
        "Version": "2012-10-17"
    }
}

Get service accounts for GCP GCS bucket

When you create a new GCS bucket in your GCP environment, you must configure proper permissions for the two service accounts associated with your Splunk Cloud Platform deployement. For more information on how to configure permissions for GCP GCS buckets, see Configure self storage in GCP.

To retrieve the two service accounts for your GCP GCS bucket, send an HTTP GET request to cloud-resources/self-storage-locations/configs/service-accounts. For example:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/configs/service-accounts' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

The response shows the two service accounts for the GCP GCS bucket. For example:

JSON 
{
    "message": "Please configure proper permissions for the GCP service accounts. Please refer to https://docs.splunk.com/Documentation/SplunkCloud/latest/Admin/DataSelfStorage for more info.",
    "serviceAccounts": {
        "clusterMaster": "indexes-acs-gcp-c0m1@indexes-acs-gcp-cdf8.iam.gserviceaccount.com",
        "indexer": "indexes-acs-gcp-idx@indexes-acs-gcp-cdf8.iam.gserviceaccount.com"
    }
}

When you configure a new self storage location for Azure, an admin must grant consent for Splunk Cloud Platform to access your Microsoft Entra ID tenant (formerly Azure AD).

Depending on whether you include the tenantId parameter in the request, you can retrieve the consent URL in two formats:

  • As a url containing a Tenant ID placeholder

    To retrieve the consent URL in this format, send an HTTP GET request, for example:
    JSON 
    curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/configs/azure-consent-url' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'

    Replace the placeholder in the retrieved URL with the Tenant ID of your Microsoft Entra ID, then use the updated URL to grant admin consent.

    Example response:
    JSON 
    {
"consentUrl": "https://login.microsoftonline.com/{tenant_id}/adminconsent?client_id=0ec7469a-f7bb-4c2f-9d7c-50d9ca99e26d",
  "message": "Use this URL to grant Azure AD admin consent. Replace {tenant_id} in the URL with your Azure AD tenant ID before opening."
}

  • As a complete URL that includes the specific Tenant ID

    To retrieve the consent URL in this format, send a HTTP GET request, for example:
    JSON 
    curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/configs/azure-consent-url?tenantId=aa4f6e43-6c53-4d2c-8f01-282fd1857897' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'
    Example response
    JSON 
    {
  "consentUrl": "https://login.microsoftonline.com/aa4f6e43-6c53-4d2c-8f01-282fd1857897/adminconsent?client_id=0ec7469a-f7bb-4c2f-9d7c-50d9ca99e26d",
  "message": "Azure AD admin consent URL is ready to use."
}

Get the Azure service principal

Splunk Cloud Platform requires the application (client) ID of the Azure service principal to authenticate securely and access your Azure self storage. To retrieve the Azure service principal, send the following HTTP GET request:

JSON 
curl 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/configs/azure-service-principal' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…'
Example response:
JSON 
{
  "clientId": "0ec7469a-f7bb-4c2f-9d7c-50d9ca99e26d",
  "message": "The Application (Client) ID of the Azure service principal used by Splunk Cloud for DDSS."
}

Configure a new self-storage location

To configure a new self storage location for DDSS, send an HTTP POST request to the cloud-resources/self-storage-locations/buckets endpoint, specifying the bucket title and bucketName parameters in the request body. You can optionally include values for folder and description parameters.

For self storage location on Azure, you must include values for tenantId and storageAccountResourceId parameters in the request body.

Before you configure a new self storage location, you must create the storage bucket in your AWS or GCP environment and apply the correct AWS IAM policy or GCP service accounts. For more information, see Store expired Splunk Cloud Platform data in your private archive in the Splunk Cloud Platform Admin Manual.

Example AWS S3 or GCP request:

JSON 
curl -X POST 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/buckets' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
   "title": "test-title",
   "bucketName": "<bucket-name>",
   "folder": "string",
   "description": "string"
}'

Example Azure request:

Note: Bucket (container) names on Azure must be 3 to 63 characters long, contain only lowercase letters, numbers, and hyphens, must start and end with a letter or number, and cannot contain consecutive hyphens (--).
JSON 
curl -X POST 'https://admin.splunk.com/{stack}/adminconfig/v2/cloud-resources/self-storage-locations/buckets' \
--header 'Authorization: Bearer eyJraWQiOiJzcGx1bmsuc2VjcmV0Iiwi…' \
--header 'Content-Type: application/json' \
--data-raw '{
  "title": "test-title",
  "bucketName": "<bucket-name>",
  "folder": "string",
  "description": "string",
  "tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "storageAccountResourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/myResourceGroup/providers/Microsoft.Storage/storageAccounts/mystorageaccount"
}

For AWS S3 storage locations, the response appears as follows:

JSON 
{
    "bucketName": "<bucket-name>",
    "bucketPath": "<bucket-name>/with-message",
    "description": "Test configuring ddss with ACS and show async message",
    "folder": "with-message",
    "title": "test-bucket-1-with-message",
    "uri": "s3://<bucket-name>/with-message"
}

For GCP GCS storage locations, the response appears as follows:

JSON 
{
    "bucketName": "<bucket-name>",
    "bucketPath": "<bucket-name>/untitled-folder",
    "description": "Test configuring ddss with ACS on GCP Stack",
    "folder": "untitled-folder",
    "title": "test-bucket-for-gcp",
    "uri": "gs://<bucket-name>/untitled-folder"
}
For Azure storage locations, the response appears as follows:
JSON 
{
  "bucketName": "<bucket-name>",
  "bucketPath": "<bucket-name>/untitled-folder",
  "description": "Test configuring Azure DDSS for archived data",
  "folder": "archived-data",
  "title": "my-azure-storage",
  "uri": "az://<bucket-name>/archived-data"
}
Note: The bucketPath value combines bucketName and folder and is used as an identifier for the self storage location.