Identify time partitions in a Microsoft Azure dataset

Identify the time partitions in your Microsoft Azure datasets to make searches more efficient and cost effective.

Note: In the Controlled Availability release stage, Splunk products may have limitations on customer access, features, maturity, and regional availability. For additional information on Controlled Availability please contact your Splunk representative.

About partitions

Partitioning is an organization strategy for large datasets that makes it possible for you to search them efficiently. When you partition your data, you organize it into a hierarchical directory structure based on the distinct values of one or more fields in the data.

For example, you might partition your application logs in Microsoft Azure by date, breaking them down by year, month, and day. Then you can place files corresponding to a single day's worth of data in a Microsoft Azure path like:

  • Hive-style: https://my_azure_storage_account.blob.core.windows.net/my_container/logs/year=2025/month=08/day=23/
  • Non-Hive: https://my_azure_storage_account.blob.core.windows.net/my_container/logs/2025/08/23/

If you have a partitioned dataset in Microsoft Azure, you can identify the time fields that make up the hierarchical structure of the data partitions. When you filter your federated searches with time partition values, those searches become more efficient and cost effective.

Note: You can identify time partition settings for a dataset even if you have not selected Define the time field. Time partition fields exist in the Amazon S3 paths that form the structure of your dataset. The Time field exists as a column in the data catalog that references your dataset.

When you define time partitions for a dataset, identify the first field in the time field hierarchy, then the second field, and so on. For example, if your data catalog references a dataset that you have partitioned by year, month, and day, identify year as the first time partition field, month as the second time partition field, and day as the third time partition field.

Define time partition settings

  1. In your dataset definition, under Time partition settings, select the Time zone that applies to your time partition fields. You must choose a Time zone if you define one or more time partition levels.
  2. Select Add field.
  3. Identify the first field by which your dataset is partitioned. This is the highest level of partitioning you use. Specify values for the following fields:
    Time partition setting Description
    Time partition field Provide the name of the time field that is the partition key for the indicated partition filter level. Values for the Time partition field can contain only lowercase letters, numbers, and underscores.
    Time format Provide a time format string for the indicated Time partition field. Compose this time format string out of Splunk-supported time format variables. For more information and examples, see Using time variables in the SPL2 Search Manual.
    Data type Select the data type of the Time partition field. Your options are String, Integer, and Date.
  4. If you have another time partition key, select Add field and identify the Time partition field, Time format, and Data type. Repeat until you have defined a partition level for each partition key you want to use in your federated searches.