Identify time partition fields among crawler-inferred partitions

After the crawler process completes for a Microsoft Azure dataset, identify time partition fields in the set of partition fields inferred by the crawler.

If you are defining a Microsoft Azure dataset and you opt to Discover partitions via crawler, understand that the crawler cannot identify time partition fields on its own. After you create the dataset and the crawler process completes, you must go to the Edit page for the dataset to identify the time partition fields in the partition field list inferred by the crawler and provide their Time zone and Time format values.
Note: The crawler process does not start until you select Create at the Review step and actually create the dataset. After the crawler process is started, it might take a few minutes to complete.
  1. After you create your Microsoft Azure dataset and the crawler process completes, open the Edit page for your dataset.
  2. In the dataset definition, under Partitions, determine whether the partition field list includes time partition fields.
  3. If you see time partition fields, go to Time partition settings and select the Time zone that applies to your time partition fields. You must choose a time zone if you define one or more time partition fields.
  4. Identify the first field by which your dataset is partitioned. Specify values for the following settings:
    Time partition setting Description
    Time partition field Provide the name of the time partition field. This must be a partition field that was inferred by the crawler. Values can contain only lowercase letters, numbers, and underscores.
    Time format Provide a time format string for the indicated Time partition field. Compose this time format string out of Splunk-supported time format variables, such as %Y for "year" values like 1980 or 2026, and %m for "month" values like 04 or 12. See Using time variables in the SPL2 Search Manual.
  5. If you have another time partition field, select Add field and identify the Time partition field and Time format of the field. Repeat this step until you have defined a level for each time partition field that the crawler inferred.
    Note: If you provide two or more time partition fields, list them in the order that the fields appear in the Amazon S3 path for the dataset. You can easily rearrange the order in which fields are listed, if necessary. For more information about the importance of partition field list order, see Identify time partitions in a Microsoft Azure dataset.
  6. Select Save to save your changes to the dataset definition.