Create an Amazon S3 dataset for federated search that is backed by an Iceberg REST catalog

Define a federated dataset that is referenced by a table in an Apache Iceberg REST catalog that you maintain.

Define an Amazon S3 federated dataset that is referenced by a table in an Apache Iceberg REST catalog that you maintain.
  • Your user account on the Splunk Cloud Platform deployment must have a role with the edit_connections and edit_datasets capabilities. See Define roles on the Splunk platform with capabilities in the Splunk Cloud Platform Manage Users and Security manual.
  • You must have completed the Select data store, Configure connection, and Define dataset steps of the Create dataset workflow. See Define an Amazon S3 dataset
  • You must have an Apache Iceberg REST catalog that refers to the dataset at the Amazon S3 location you supplied in the Define dataset step.
  • The dataset must have an Amazon S3 connection definition that includes the Catalog URL of your Apache Iceberg REST catalog. See Create an Amazon S3 connection.
  1. On your Splunk Cloud Platform deployment, in Catalog, at the Configure dataset step of the Create dataset workflow, select Apache Iceberg REST as the type of catalog you use to represent your dataset.
  2. Specify the following settings for the Apache Iceberg REST catalog table that references your dataset.
    Setting Description
    Catalog name A unique, user-defined name for the specific Iceberg catalog instance you are connecting to.
    Table namespace A logical grouping that organizes a set of tables within the Iceberg REST catalog, similar to databases or schemas in traditional SQL systems.
    Table namespaces are hierarchical. A table namespace can be a top level namespace, such as sales, or from a lower level in the namespace hierarchy separated by dot characters, such as finops.billing.west_region.
    Table name The specific name for the Iceberg catalog table that you want to run federated searches over. The Table name points to the dataset stored at the Amazon S3 location and must belong to the Table namespace.
  3. (Optional) Select Define the time field if this dataset contains time-series data and you plan to use time-based filtering or SPL2 time functions when you run federated searches over it.
  4. If you select Define the time field, fill out the Time settings: Time field, Time format, and Unix time field.
    For more information, see Identify the time field in an Amazon S3 dataset.
  5. If your dataset is partitioned into data subsets by time, identify Time partition fields to improve search performance and reduce search cost.
    For more information, see Identify time partitions.
  6. Select Next to move on to the Update policies step.
Authenticate access to your Amazon S3 federated dataset by applying a Splunk-generated resource access policy to the AWS IAM role that is associated with the dataset's connection. See Apply the dataset resource access policy to an AWS IAM role.