Define a CloudWatch Unified Data Store dataset

Define a CloudWatch Unified Data Store federated dataset in Catalog to facilitate federated search of CloudWatch Unified Data Store data stored in a specific Amazon S3 table.

In Catalog, define an Amazon CloudWatch Unified Data Store federated dataset for use in federated searches. Each CloudWatch Unified Data Store dataset you define lets you run federated searches over CloudWatch data stored in a specific Amazon S3 table.

  • You must have a Splunk Cloud Platform (SCP) deployment that is hosted on AWS (Amazon Web Services).

  • Your user account on the SCP deployment must have a role with the edit_connections and edit_datasets capabilities. See Define roles on the Splunk platform with capabilities in Securing Splunk Cloud Platform.

  • Use the Catalog to create and maintain your CloudWatch Unified Data Store federated dataset. See Discover Data Using the Catalog.

  • You must have an Amazon Web Services (AWS) account and an AWS IAM role with permissions that let you attach and modify custom trust policies and permissions policies for IAM roles. Contact your AWS administrator for assistance with AWS permissions. See IAM role creation in the AWS Identity and Access Management User Guide.

  • You must have Amazon S3 Tables Integration set up in your AWS account, and you must use it to set up managed Amazon S3 Tables that are linked to your CloudWatch Logs data sources.

    • Use Amazon S3 Tables Integration to associate specific CloudWatch log sources with Amazon S3 Tables based on data source name and type.

    • Amazon S3 Tables use the Apache Iceberg format, which lets you run Splunk Federated Search queries over the CloudWatch Logs data they're associated with.

    • See Access logs with S3 Tables Integration in the Amazon CloudWatch Logs User Guide for details.

  1. On your Splunk Cloud Platform deployment, in Splunk Web, select the Catalog icon ( This image shows the Catalog icon, which looks like a document catalog.) to open the Catalog.
  2. On the Datasets tab, select Create > Dataset to enter the Create dataset workflow.
  3. On Get started, select CloudWatch Unified Data Store. Then select Next.
  4. On Configure connection, determine whether you want to use an existing CloudWatch Unified Data Store connection or create a new CloudWatch Unified Data Store connection.
    • If a suitable CloudWatch Unified Data Store connection exists that you want to associate your dataset with, select Associated connection, and choose an existing CloudWatch Unified Data Store connection from the drop-down list. If the connection you select is ready to be used and its details are correct, select Next.
    • If no CloudWatch Unified Data Store connection exists that you want to associate you dataset with, select Create connection to define a new connection for your dataset. See Create a CloudWatch Unified Data Store connection. When you have successfully created a new CloudWatch Unified Data Store connection, select Next.
  5. On the Define dataset page, provide values for the following fields, and then select Next.
    Settings Description
    Dataset name Supply a unique name for your dataset. The dataset name can contain only alphanumeric characters, underscores, and hyphens.
    Dataset description (Optional) Provide a description for your dataset.
    Note: The S3 Table bucket name is pre-configured for CloudWatch Unified Data Store datasets and you do not need to set it.
  6. On Configure dataset, enter the S3 table that is associated with the CloudWatch data that you want to search. The Amazon S3 table must be contained by the Table database and Catalog name that Splunk software has identified for this dataset.
    Note: To search CloudWatch data in an Amazon S3 table you must first use S3 Tables Integration to set up the S3 table and associate it with your CloudWatch log data sources. See Access logs with S3 Tables Integration in the Amazon CloudWatch Logs User Guide.

    To locate your CloudWatch S3 Tables and view their details, go to your Amazon S3 console, select Table buckets, select the aws-cloudwatch bucket, and then select the name of the S3 table that you want to review.

  7. (Optional) Select Define the time field if your dataset contains time-series data and you intend to use time-based fields and functions when you run searches over it.
    Note: If you have a time field in your table or view and you do not define it, searches of this dataset that have time range filters return incorrect results.

    If you select Define the time field, you must identify the Time field, Time format, and UNIX time field. These settings identify the time field in your dataset, provide its time format, and indicate the UNIX time field alias you want to use in your searches.

    For more information about the Time settings fields, see Identify the time field in a federated dataset.

  8. Select Next.

Go to the Update policies step, where you'll finish creating your CloudWatch Unified Data Store federated dataset by applying a new resource access policy to the IAM role that is associated with your dataset's connection.

Additionally, if access to the S3 Table that holds your dataset is governed by Lake Formation permissions, you'll grant additional permissions to that same IAM role.

See Set the access policy and permissions for a CloudWatch Unified Data Store dataset.