Manually identify time partition fields for a dataset with a Glue or Iceberg catalog

Manually identify time partition fields for Amazon S3 datasets backed by Amazon Glue or Apache Iceberg REST data catalogs.

If you are defining an Amazon S3 dataset backed by an Amazon Glue catalog table or an Apache Iceberg REST data catalog, add time time partition fields in the Time partition settings section.

  1. Under Time partition settings, select the Time zone that applies to your time partition fields. You must choose a time zone if you define one or more time partition fields.
  2. Identify the first time field by which your dataset is partitioned. Specify values for the following settings:
    Time partition setting Description
    Time partition field Provide the name of the time partition field. Note that this must be a field that appears in your schema. Values can contain only lowercase letters, numbers, and underscores.
    Time format Provide a time format string for the indicated Time partition field. Compose this time format string out of Splunk-supported time format variables, such as %Y for "year" values like 1980 or 2026, and %m for "month" values like 04 or 12. See Using time variables in the SPL2 Search Manual.
    Data type Identify the data type of the time partition field.
  3. If you have another time partition field, select Add field and identify the Time partition field and Time format of the field. Repeat this step until you have defined a level for each time partition field that the crawler inferred.
    Note: If you provide two or more time partition fields, list them in the order that the fields appear in the Amazon S3 location path for the dataset, as specified in Identify time partitions. You can easily rearrange the order in which fields are listed by dragging fields above or below other fields.
  4. When you are finished adding fields, select Next to save your changes and go to the Update policies step.
Go to the Update policies step. See Apply the dataset resource access policy to an AWS IAM role.