Archive your metrics to reduce billing cost

Archive your metrics to reduce overall billing cost.

Introduction to archived metrics

You can scale your metric data by sending and storing low-value, infrequently accessed metrics in a cheaper archival tier. Metrics stored in archived metrics are kept, but you cannot use them in charts or detectors directly.

Note: Archived Metrics cost one-tenth of real-time metrics.

If you need to use a metric that you've sent to archived metrics you can route it back to real-time metrics and access it in charts or detectors. You can also backfill historical data from up to 31 days and restore it to the real-time tier if you need to.

Use pipeline automation to automatically archive unused metrics

Pipeline automation automatically determines whether your metrics aren’t being used and archives the unused metrics, keeping them in a lower-cost storage. You can bring archived metrics back into real-time storage using metrics pipeline management exception rules.

To activate pipeline automation, follow these steps:

In Splunk Observability Cloud, select Settings > Metrics Management.
Select the Pipeline automation tab. This opens the pipeline automation menu.
Enter the age for which unused metrics will be archived, and enter a new metric grace period. To learn more about these fields, see Unused metrics in pipeline automation and Customize the new metric grace period in pipeline automation, respectively.
(Optional) Preview your pipeline automation rule. Before activating automated archival, you can create a preview showing which metrics will be archived. To see this preview, select Preview. The preview will be saved to your browser as a comma-separated values (CSV) file.
Select Activate automated archival > Activate.

Note: To activate pipeline automation and change configurations, you must be an administrator in your org. You can still preview unused metrics and add metrics to the exemption list if you have the power role. To learn more, see Create and manage teams in Splunk Observability Cloud.

Unused metrics in pipeline automation

By default, Splunk Observability Cloud declares a metric “unused” when it isn’t referenced in any charts or detectors and hasn’t been called in an API for 60 days. Splunk Observability Cloud automatically archives any metrics that it declares as “unused”.

You can change the number of days after which a metric is considered unused to 30 days, 45 days, or 60 days. To do so, follow these steps:

In the pipeline automation menu, select the box under "Age of unused metric". This opens a dropdown menu.
In the dropdown menu, select the desired number of days (30, 45, or 60).
Select Apply.

Customize the new metric grace period in pipeline automation

By default, new metrics are given a grace period of 60 days. If the metric is flagged as “unused” after this grace period ends, Splunk Observability Cloud will consider the new metric for automated archiving.

You can customize the number of days for which this grace period lasts. To do so, follow these steps:

In the pipeline automation menu, select the box under “New metric grace period”. This opens a dropdown menu.
In the dropdown menu, enter the number of days (0, 30, 45, or 60).
Select Apply.

Note: If the new metric grace period is set to 0 days, all new metrics will immediately be considered for automated archiving. This can greatly reduce usage, but requires users to create exception rules if they want to use new metrics.

Use the automation exemption list for pipeline automation

You can add metrics to an Automation exemption list. This prevents pipeline automation from archiving these metrics, even if they become unused.

To do this, search for the name of your metric under “Automation exemption list”, and select Add metric.

Note: The automation exemption list doesn’t prevent users from manually archiving metrics. If you add a metric that’s already archived or dropped to the automation exemption list, they will remain archived or dropped. If you want to change the routing of these metrics, an admin must do so manually

Deactivate pipeline automation

To deactivate automated archiving, select Deactivate automated archival. Then, select Deactivate.

When deactivating automated archival, all currently archived metrics will stay archived. You can create exception rules to restore previously archived data. To learn more, see Use routing exception rules to route a specific MTS or restore archived data.

Create exception rules for archived metrics

Archived metrics might not appear in your charts, detectors, and dashboards. To make archived metrics visible, you can create exception rules that restore the archived data and display it in your charts.

You can create exception rules by using the in-context modal or by using metrics pipeline management tools.

If your chart contains override filters or if you're using the chart in an APM detector, then you must use metrics pipeline management to create exception rules. Metrics starting with ^gcp , ^aws, and ^azure can't be restored using the in-context modal. Use metrics pipeline management to restore these metrics.

Follow these steps to restore archived data with the in context modal:

In your dashboard, select a chart that contains archived metrics. Charts with archived metrics will display a yellow warning symbol.
In the warning banner, select Show archived metrics. This opens the archived metrics modal. The modal displays which archived metrics you can restore under "Show incoming data".
Under "Show historical archived data", select a time range from which to restore data. The maximum time range is 1 month, or 31 days.
The modal displays approximately how many MTS it will restore. Select "confirm" to begin restoration.

Use metrics pipeline management to create exception rules

You can also use metrics pipeline management to manually create exception rules that restore archived data. To learn more, see Use data routing to keep, archive, or discard your metrics.

Documentation