Apply the dataset resource access policy to an AWS IAM role

Apply the Splunk-generated resource access policy to the IAM role associated with your connection to authenticate access to your Amazon S3 dataset.

To authenticate access to your Amazon S3 dataset, you must apply a Splunk-generated resource access policy to the IAM role associated with the dataset's connection. This resource access policy is generated by Splunk software. It controls the following:

  • The resource access policy grants access to the Amazon S3 bucket that contains the Amazon S3 location you specified for the dataset in the Define dataset step. See Define an Amazon S3 dataset.
  • If your Amazon S3 dataset is backed by an AWS Glue catalog table, the resource access policy grants access to the AWS Glue catalog that contains the Glue table. See Create an Amazon S3 dataset for federated search that is backed by an AWS Glue catalog table.
  • If your Amazon S3 dataset is backed by a Splunk-native data catalog, and you have set up an SQS queue and event notification for the Amazon S3 bucket that contains your dataset, the resource access policy contains the SQS queue ARN that you supplied earlier in the dataset definition. This ARN enables the Splunk-native catalog to stay in synch with your dataset as its contents change. See Set up automated updates for Splunk-native data catalogs in AWS.
  • If you are using server-side encryption (SSE-KMS) to encrypt data in the Amazon S3 bucket that contains your dataset or the AWS Glue catalog table that refers to it, the resource access policy allows Splunk software to access that data.
  • Your user account on the Splunk Cloud Platform deployment must have a role with the edit_connections and edit_datasets capabilities.
  • Review the connection this dataset is associated with. Obtain the name of the IAM role that it uses for IAM role authentication. See Define an Amazon S3 dataset.
  • You must have completed the Configure dataset step of the Create dataset workflow.
  1. On your Splunk Cloud Platform deployment, in the Data Management app, at the Update policies step of the Create dataset workflow, update the AWS KMS key ARNs field with AWS KMS key ARN values for your Amazon S3 bucket or AWS Glue data catalog, if you have provided server-side encryption with AWS key management service keys for either object.
    For instructions on obtaining these keys, go to the following links:
    Note: Skip this step if you have not applied SSE-KMS encryption to your bucket or your AWS Glue data catalog.
  2. In a new browser tab, log in to your AWS account and navigate to the Identity and Access Management (IAM) console.
  3. In the left-hand navigation pane of the IAM console, select Roles.
  4. In the Roles list, select the name of the role that is used for IAM role authentication by the connection this dataset is associated with.
  5. In the Permissions policies section of the role, select Add permissions > Create inline policy.
  6. On the Specify permissions page, in the Policy editor, select JSON.
  7. Back on the browser tab that contains your federated search dataset definition, select Copy for your Resource access policy.
  8. Return to the browser tab that displays the IAM console and paste the resource access policy into the Policy editor, overwriting the sample policy as you do so.
    Note: Resolve security warnings, errors, general warnings, and suggestions before saving your policy.
  9. Select Next.
  10. On the Review and create page, do the following things:
    1. In the Policy details section, give your policy a unique and easily identifiable Policy name.
    2. Select Create policy.
  11. Back on the browser tab that contains your federated search dataset definition, select Next.
  12. On the Review page, review your dataset definition. If the details appear correct, select Create dataset to create your dataset.

Your Amazon S3 dataset is created or is in the process of being created.

On the Datasets listing page you can see the Status of your Amazon S3 dataset, and you can use that status value to guide your next actions regarding it.

Status Description Action
Ready The dataset is available for use in federated searches.
Processing The crawler process is running over the dataset.

If your Amazon S3 dataset is backed by a Splunk-native data catalog and you have selected Discover schema via crawler or Discover partitions via crawler during the Configure dataset step of dataset definition, selection of Create dataset on the Review step causes the crawler process to initiate schema and parititon field discovery for the dataset.

The crawler process might take a few minutes to complete.

Note: If more than 10 minutes pass and the crawler process is still in Processing status, a dataset setup error might be causing it to fail to complete. Review the current dataset configuration for errors such as an incorrect location path. Then delete the dataset that is stuck in Processing status and try to recreate it without errors.
Needs action The schema and partitions discovered by the crawler require review and confirmation.

Go to the Edit page for your Amazon S3 dataset. Review the schema and partition fields that the crawler has discovered, make edits as necessary, and confirm that you have reviewed the discovered fields. See Review the crawler-discovered schema and partitions for an Amazon S3 dataset.
Error An error occurred during dataset creation or processing. Review the configuration for your Amazon S3 dataset, correct issues, and recreate the dataset if necessary.