Manage metric rollup policies with configuration files

If you have access to the configuration files for your deployment, you can manually configure metric rollup policies for your source metric indexes.

See Roll up metrics data for faster search performance and increased storage capacity for a conceptual overview of metric rollup policies.

Note: You should have already identified or created a source metric index and one or more target metric indexes before you create a metric rollup policy configuration. These indexes must be discoverable on the search head. If you use distributed search you have to create stand-in indexes and set up data forwarding to enable metric rollup policies.

See Index prerequisites for metric rollup policies.

Create metric rollup policies that are shared with apps other than Search & Reporting

By default, the Splunk software gives metric rollup policies that you create with Splunk Web the context of the Search & Reporting app. If you are manually configuring your metric rollup policies, you can create metric rollup policies for other apps by adding a metric_rollups.conf file to into the etc/apps/<app-name>/local directory for the app and then putting the configuration for the rollup policy in that file. This file placement creates a metric rollup policy that is owned by "nobody" and shared to all users of the app.

Note: You cannot create a metric rollup policy that is private or owned by a specific user. Metric rollup policy configurations created in etc/users/<user-name>/<app-name> are ignored by the Splunk software.

App-specific rollup policies generate the scheduled searches that populate their rollup summaries in a savedsearches.conf file in etc/apps/<app-name>/local. The app context of these saved searches is included in the names of their configuration stanzas, which double as their object name. The names of these searches fit the following syntax: _ss_mrollup_<source_index>_<span>_<target_index>_<app_name>.

Metric rollup feature extensions that are not available in Splunk Web

When you manage metric rollup policies through direct edits to configuration files, you can take advantage of optional feature extensions that are not yet available in Splunk Web.

Extended feature In Splunk Web Through configuration file edits Setting
Change the default aggregation function The default aggregation function for a new rollup policy is fixed to avg. There is no way to change this in Splunk Web. You can change the default aggregation function to one or more aggregation functions. The search head rolls up aggregated metrics for each function you specify. defaultAggregation
Metric filtering You cannot design a rollup policy that filters specific metrics out of the resulting rollup summaries. You can identify a set of metrics that are specifically included in or excluded from the summaries generated by the rollup policy. metricList and metricListType
Multiple aggregation functions for metric exclusion rules You can specify only one alternate aggregation function for a metric. You can define exclusion rules that specify two or more alternate aggregation functions for a metric. aggregation.<metric_name>

You can also manage this extended functionality through REST API operations on the metric rollup endpoints. See Metrics Catalog endpoint descriptions in the REST API Reference Manual.

Specify a metric rollup policy stanza in metric_rollups.conf

To configure a metric rollup policy you need to add a stanza to your metric_rollups.conf file.

The configuration syntax for a metric rollup policy stanza is as follows: 

[index:<Metric Index Name>]
defaultAggregation = <'#' separated list of aggregation functions>
rollup.<summary number>.rollupIndex = <string Index name>
rollup.<summary number>.span = <time range string>
metricList = <comma-separated list of metrics>
metricListType = <excluded/included>
dimensionList = <comma-separated list of dimensions>
dimensionListType = <excluded/included>
aggregation.<metric_name> = <'#' separated list of aggregation functions>

The following table defines these settings. It explains which settings are required and which are optional.

Setting Value Required? Description Default Value
[index: <Metric Index Name>]A name of a source metric index.YesThis is the stanza header. It names the source metric index to which the metric rollup policy belongsn/a
defaultAggregationA list of aggregation functions, separated by # characters.YesThis setting provides the set of aggregation functions that the rollup search uses when it aggregates the metric data points in the source metric index for a rollup summary. The defaultAggregation can be overruled for specific metrics by the aggregation.<metric_name> setting. This setting supports the following functions: avg, count, max, median, min, perc<int>, and sum.avg
rollup.<summary number>.rollupIndex The name of the target index for the rollup. Yes This setting is one half of a rollup summary definition. Both halves of a rollup summary definition should have the same <summary number>.LB, LBThe <string Index name> is the name of the target metric index where the summary is stored. This setting is required. Do not leave it blank.LB, LBA metric rollup policy can have multiple rollup summary definitions. To be valid, a metric rollup policy stanza must include at least one full rollup summary definition. This setting is required. Do not leave it blank. summary number = 1, string index name = Metric Index Name from header
rollup.<summary number>.spanA time range string.YesThis setting one half of a rollup summary definition. Both halves of a rollup summary definition should have the same <summary number>

The <time range string> is the period of the scheduled search that populates the rollup summary with rolled up metric data points that are aggregations of metrics in the source index

The rollup <time range string> is limited to the following values for minutes, hours, and days. Other time units are not allowed.

m (minutes) Valid values: 1, 2, 3, 4, 5, 6, 10, 12, 20, 30, 60

h (hours) Valid values: 1, 2, 3, 4, 6, 8, 12, 24

d (days) Valid value: 1

rollup.<summary number>.span has a lower boundary governed by the minspanallowed setting in limits.conf. minspanallowed defaults to 300 seconds, or five minutes

A metric rollup policy can have multiple rollup summary definitions. To be valid, a metric rollup policy stanza must include at least one full rollup summary definition. This setting is required. Do not leave it blank.

summary number = 1, time range string = 1h
metricListA comma-separated list of metric names. No Use this setting in conjunction with metricListType to create a filter at the search head that allows certain metrics to be rolled up but not others. All of the listed metrics should have metric data points in the source metric index. Empty string
metricListType [included | excluded]No Use this setting in conjunction with the metricList setting to create a filter at the search head that allows certain metrics to be rolled up but not others. When you set metricListType to excluded, the search head rolls up all available metrics from the source index except the metrics in metricsList. When you set metricListType to included, the search head rolls up only the metrics in metricsList and filters out all other metrics in the source index. excluded
dimensionListA comma-separated list of dimensions. No Use this setting in conjunction with dimensionListType to create a filter at the search head that allows certain dimensions to be rolled up but not others. All of the listed dimensions should appear in metric data points in the source index. Empty string
dimensionListType[included | excluded]No Use this setting in conjunction with the dimensionListsetting to create a filter at the search head that allows certain dimensions to be rolled up but not others. When you set dimensionListType to excluded the rollup metrics produced by the rollup policy include all available dimensions in the source index except the ones in the dimensionList. When you set dimensionListType to included, the rollup metrics produced by the rollup policy filter out all available dimensions except the ones in the dimensionList. excluded
aggregation.<metric_name>A list of aggregation functions, separated by # characters. No Use this optional setting to provide an exclusion rule for a specific metric_name in the source metric index. An exclusion rule defines a separate set of aggregation functions for the metric_name.

Use it to specify a different set of aggregation functions for that metric_name. A metric rollup policy can have multiple exclusion rules as long as they are each for a different metric_name. Do not set up an exclusion rule that uses the same set of aggregation functions as the defaultAggregation setting. This setting supports the following functions: avg, count, max, median, min, perc<int>, and sum.

Empty string

Change the minimum span allowed for a rollup summarization search

The rollup.<summary number>.span setting has a lower boundary that is determined by the minspanallowed limit for the [rollup] stanza in limits.conf. minspanallowed is set to 300 seconds, or 5 minutes, by default. If you provide a span for a rollup summarization search that is lower than minspanallowed, you will see an error message.

This limit is meant to prevent you from setting up rollup summarization searches with a frequency that would likely lead to search concurrency problems, where scheduled searches fail to run when they should because there are too many searches running at once. However, if you need to change this limit, you can. Do not set minspanallowed to a value lower than 60 seconds.