Maximum number of metric time series processed in a signal
To maximize responsiveness when processing incoming metric data, Splunk Observability Cloud limits the number of metric time series (MTS) processed in a signal (single plot line) on a chart, detector, or navigator. If the total number of time series for the metric specified in your signal exceeds this limit, Splunk Observability Cloud displays a message at the top of the chart, detector, or navigator. This message notifies you that the processed data reflects only a subset of the time series in the data.
To ensure that your charts, detectors, and navigators are processing all the time series in the data, apply as many filters as necessary to make the MTS number less than the limit. You'll know you've succeeded when the message at the top of the chart, detector, or navigator no longer appears.
For example, suppose you want to display the sum of available disk capacity per availability zone. You might use df.complex.free as your signal, then apply the Sum analytics function, grouped by availability zone. If you have a high number of disks, the number of MTS returned by this signal might exceed the limit. To reduce the number of MTS, use filters and multiple plots instead of the group-by option in the Sum analytics function.
Use filters and multiple plots on charts and detectors
-
Apply a filter for the plot that selects a single availability zone, then use the Sum analytic function without the group-by option.
-
At the end of the plot row, select Configure plot to open Visualization Options, then select a color from Plot Color.
-
Clone the plot. To do this, open the Plot actions menu at the end of the plot row in the Plot Editor, then select Clone.
-
In the cloned plot, change the availability zone filter to select another availability zone, and change the Plot Color to give the cloned plot a color that's different from the original plot.
-
Continue to clone plots, change the availability zone filter, and change the plot color until you have one plot for each availability zone.
The result is one chart with a separate plot line for each availability zone. Each plot line has a distinct color.
For more information on filtering, see Filter the signal.
Add a filter on a navigator
To add filter on a navigator, select Add Filter to create a filter and view a specific slice of your environment based on dimensions or properties you specify. Filtering is particularly useful for viewing only the instances running a specific service, or in a particular availability zone.
Filters that you apply to your host instances also filter dependencies in the navigator sidebar. To learn more about the navigator sidebar, see View dependencies in the navigator sidebar in the same topic.
Configure a navigator to require a filter to display data
When a navigator reaches the maximum number of metric time series (MTS) processed in a signal, it displays only a subset of the data until you apply a filter to limit the results. For example, you can apply the cluster.name filter to a Kubernetes navigator to only process data from a specified cluster. For more information on adding filters to navigators and MTS limits, see Add filter and Maximum number of metric time series processed in a signal.
MTS limit violations can also cause low responsiveness in navigators. To prevent this, you can configure a navigator to require a filter in order to display data. When you configure this setting, the navigator will not display data on the summary card on the Infrastructure landing page or on the aggregate views (overview, table view, and heat map view) until a filter is added.
To configure a navigator to require a filter to display data:
-
From the Splunk Observability Cloud main menu, select .
-
Select a navigator.
-
Select the settings icon in the upper-right corner.
-
Select Manage navigator settings.