Create an Amazon S3 connection

Create an Amazon S3 connection in the Data Management app to authenticate federated searches over Amazon S3 datasets from your Splunk platform deployment.

Create an Amazon S3 connection in the Data Management app to authenticate federated searches over Amazon S3 datasets from your Splunk platform deployment.

Begin by determining the data store of the Amazon S3 connection, identifying the AWS account that it connects to, and indicating what abilities it supports. Then copy a Splunk-generated custom trust policy and use that policy to set up IAM role authentication in your AWS account, to enable federated searches of your Amazon S3 data. Finally, optionally configure Apache Iceberg REST catalog authentication and create your connection.

  • Your Splunk Cloud Platform deployment must be on version 10.4.2604 or higher.
  • Your user account on the Splunk Cloud Platform deployment must have a role with the edit_datasets and edit_federated_providers capabilities. See Define roles on the Splunk platform with capabilities in the Splunk Cloud Platform Manage Users and Security manual.
  • You must have an Amazon Web Services (AWS) account and an AWS IAM role with permissions that let you attach and modify custom trust policies and resource policies for IAM roles. Contact your AWS administrator for assistance with AWS permissions. See IAM role creation in the AWS Identity and Access Management User Guide.
  1. On your Splunk Cloud Platform deployment, in Splunk Web, open the Data Management app.
  2. Select Connections > Create connection to enter the Create connection workflow.
  3. On Select data store, select Amazon S3. Then select Next.
  4. On General, provide values for the following settings, and then select Next.
    Setting Description
    Connection name Give this connection a name. The connection name can contain only alphanumeric characters, underscores, and hyphens.
    Description (Optional) Provide a description of the connection.
    AWS account ID Enter the 12-digit AWS account ID for the AWS account that contains the datasets you want to search.
    AWS region Select the AWS region for the AWS account.
  5. On Select abilities, select Run Federated Search.

    You can optionally select Send data from Ingest Processor or Send data from Edge Processor and choose their authentication methods if you want to use this connection to send data from your Splunk Cloud Platform deployment to the same Amazon S3 datasets that you run federated searches over.

    For details about setting up a connection that supports sending data to Amazon S3 datasets with Edge Processor, see Create an Amazon S3 connection for Edge Processor pipelines in Use Edge Processors for Splunk Cloud Platform.

    For details about setting up a connection that supports sending data to Amazon S3 datasets with Ingest Processor, see Create an Amazon S3 connection for Ingest Processor pipelines in Use Ingest Processors.

  6. Select Next to go to the Storage authentication step.
Set up IAM role authentication in your AWS account:
  1. At the Storage authentication step, select Copy for the Custom trust policy to copy the trust policy to your clipboard.
    You can optionally open the custom trust policy window to review the generated trust policy.
  2. In a new browser tab, log in to your AWS account and navigate to the Identity and Access Management (IAM) console.
  3. In the left-hand navigation pane of the IAM console, select Roles and then select Create role.
  4. On the Select trusted entity page, in the Trusted entity type section, select Custom trust policy.
  5. In the custom trust policy editor, replace the example custom trust policy with the custom trust policy you copied from your Splunk deployment. Then, select Next.
  6. On the Add permissions page, do not make any changes. Select Next.
  7. On the Name, review, and create page, do the following things.
    1. Give the role a Name that is unique within your AWS account.
      Role names are not distinguished by case, meaning that you cannot simultaneously have roles named MyRole and myrole.
    2. Review the Trust policy that you added in the previous steps to ensure that it is correct.
    3. In Add tags, add a resource tag with a Key of splunk-assumable-role and a Value of true.
    4. Select Create role.
  8. Find the role you just created and select the role name to open it.
  9. Select the copy icon to copy the role ARN.
  10. Go back to your first browser tab then do the following things: and paste the role ARN value into the IAM role ARN field for the Data Management app.
    1. Paste the ARN into the IAM role ARN field.
    2. Select I confirm that I have added the tag to the new IAM role.
  11. Depending on whether you previously selected other abilities or authentication methods on the Select abilities page for this connection, do one of the following:
    Option Description
    You have only selected the Run federated search ability. Select Next to proceed to the Catalog authentication page.
    You also selected the Send data from Edge Processor ability, the Send data from Ingest Processor ability, or both, and selected IAM role as the authentication method for all abilities. Select Next to proceed to the Catalog authentication page.
    You also selected the Send data from Edge Processor ability and selected Access key as the authentication method for the additional ability. You must configure additional authentication settings. See Authenticate an Amazon S3 connection for Edge Processor using access keys in the Use Edge Processors for Splunk Cloud Platform manual.
    You also selected the Send data from Ingest Processor ability and selected Access key as the authentication method for the additional ability. You must configure additional authentication settings. See Authenticate an Amazon S3 connection for Ingest Processor using access keys in the Use Ingest Processors manual.
Authenticate your Apache Iceberg REST catalog:
  1. At the Catalog authentication step, indicate whether you use an Apache Iceberg REST catalog to represent your data.
    Note: If you do not use an Apache Iceberg REST catalog, select No for Do you use an Apache Iceberg REST catalog to represent your data? and select Next. Skip to the last step of this procedure.
  2. In Catalog URL, provide the URL of your Apache Iceberg REST catalog account endpoint.
  3. Select No catalog authentication required.
    Note: OAuth2 catalog authentication is currently not supported.
  4. Select Next to go to the Review page for your connection.
  5. If the connection is defined correctly, select Create to create the connection.
After you create a connection, define a dataset for that connection. See Define an Amazon S3 dataset.