Feature preview: Cisco Deep Time Series Model

The Cisco Deep Time Series Model (CDTSM) provides forecasting, predictive alerting, and anomaly detection with no model training or upfront effort required.

CDTSM is a pretrained, generative AI model designed for forecasting metric time series data across the Splunk platform. This allows you to do predictive analytics and anomaly detection on any metric time series, from any source.
Note: Participation in this preview is optional. You can provide feedback on this preview at https://voc.splunk.com/preview/ctsm_aitk
Unlike other machine learning (ML) models, no per-metric training is required. CDTSM can recognize patterns, trends, and seasonal cycles in a time series' history and make forecasts of how that time series might continue into the near future.
Beginning with version 5.7.3, the CDTSM powers accurate, reliable anomaly detection in metric time series data.
Note: The CDTSM model powers the built-in anomaly detection logic by providing forecasts and probability quantiles that predict how the metric time series is expected to behave.
CDTSM offers time series analysis and anomaly detection without the need for data science expertise. The provided model can generate forecasts, and look for anomalies in those forecasts, without requiring custom training or fine-tuning on your specific data, making it immediately usable for predictive analytics.

Preview disclaimer

Beta features described in this document are provided by Splunk to you "as is" without any warranties, maintenance and support, or service-level commitments. Splunk makes this Beta feature available at its sole discretion and may discontinue it at any time. Use of Beta features is subject to the Splunk Pre-Release Agreement for Hosted Services.

Requirements

Splunk Cloud platform customers must run AI Toolkit version 5.7.3 in a supported region to try the preview of the Cisco Deep Time Series Model (CDTSM). All AWS regions are supported. CDTSM is one of Splunk's Cloud Hosted AI model's, running on dedicated GPU servers separate from your stack.

Splunk on-premises users must run AI Toolkit version 5.7.3, and install an instance of the open-source Cisco Time Series Model. The model can run on either CPUs or GPUs.

For more information see Getting started.

Usage and metering

There is no additional cost for using the Cisco Deep Time Series model. During the current beta period, there is a rate limit of 50 model requests per minute. This is managed by the AI Toolkit.

The AI Toolkit will use batching for larger queries. If you find that you are being rate limited, contact your account team, or make a request in the #artificial-intelligence channel in the Splunk Community Slack workspace. See http://splunk-usergroups.slack.com/.

Use cases for the CDTSM

The following are common use cases for the Cisco Deep Time Series Model (CDTSM).

Forecasting use cases:

Use case Description
Predictive Analytics Predict performance and utilization metrics for operations support, within the next 30 hours​. Metrics can be derived from log queries or metric store.
Predictive Alerting Prevent incidents and get alerted when a KPI is forecasted to exceed a threshold, with enough time to resolve the problem before service is disrupted. Metrics can be derived from log queries or metric store.

Anomaly Detection use cases:

Use case Description
Anomaly Alerting Prevent incidents and get alerted of unexpected changes in user experience or in the behavior of your service. Resolve changes in trends before they escalate.​
Dashboards Display time series anomalies on a dashboard that is used in a NOCC or SOC.
Troubleshoot Troubleshoot incidents by finding all potentially related anomalies in a stack and distributed transactions.​

Cisco Deep Time Series Model Showcase

There are 2 Showcases through which you can see the Cisco Deep Time Series Model (CDTSM) in action. From within the AI Toolkit select the Showcase tab and the Cisco Deep Time Series Model tile.

Choose the Forecasting or Anomaly Detection option as shown in the following image:

This image shows the AI Toolkit and the view from within the Showcase tab. A new tile labeled Cusco Deep Time Series Model is highlighted.

Getting started

Complete the following steps depending on if you are a cloud or on-premises customer.

Cloud customers

For Cloud customers, after installing or upgrading to AI Toolkit version 5.7.3 from Splunkbase, create a time series using SPL, with a command such as timechart, and add apply CDTSM <fieldName> to the end of the SPL search. There is no other model related configuration required.

Note:

Your user role must have the permission list_tokens_scs.

On-premises customers

For on-premises customers, after installing or upgrading to AI Toolkit version 5.7.3, you must install the open-source Cisco Time Series Model on a server that your Splunk Deployment can access. <url>Navigate to our GitHub repository for instructions.</url>. Then you must update the .conf file as defined below, and on the GitHub repository.

Forecasting syntax and parameters

Follow this syntax to call the Cisco Deep Time Series Model for forecasting:

CODE 
apply CDTSM <fields_to_forecast> [time_field=<str>] [forecast_k=<int>] [quantiles=<str>][conf_interval=<int>] [holdback=<int>] [show_input=<bool>]
To start, use apply CDTSM <field_to_forecast>. All other parameters are optional.
You can include the following arguments and parameters when using the CDTSM for forecasting:
Name Type Description
fields_to_forecast argument Space separated fields, depicting the time series fields.
Note: Wildcards (*) are not supported.
time_field parameter The field containing timestamp information.
forecast_k parameter The number of future timestamps that need to be forecasted. Default value of 128.
holdback parameter The number of input data points to be held back from the model. Useful to compare ground truth and predictions. The holdback cannot be greater than forecast_k. When you specify forecast_k, you must also account for the holdback value. Default value of 0.
conf_interval parameter This is the confidence interval in percentage around forecasted values. By default it is set to 90%. Permissible values are 20, 40, 50, 60, 80, 90, and 98.
quantiles parameter The comma separated integers signifying any explicit quantile values that you want to see in forecasts. Permissible values are mean, p1, p10, p20, p25, p30, p40, p5, p50, p60, p70, p75, p80, p90, p95, p99.
Note: You must use quotes when passing in multiple comma separated values.
show_input parameter To see the input data as well in result, set it to true. When set to false will only show holdback and forecast_k datapoints in output.

Forecasting syntax constraints

Consider the following constraints when forecasting:

  • You can exclude both the holdback and forecast_k parameters. But you cannot specify both with a value of 0 in the ML-SPL search.

  • The value for holdback is 0 by default and cannot be a negative number.

  • The value for holdback can't be greater than the input value. Input minus the holdback is the number of datapoints fed into the model.

  • The value for holdback cannot be greater than forecast_k.

  • The model supports up to 30 thousand datapoints. If you pass in more than 30 thousand datapoints the model uses the most recent 30 thousand datapoints.

  • You must apply quotes when passing in multiple comma separated values.

  • Intervals can be anything other than 0 seconds.

Using Forecast visualizations

After your search completes, select the Visualization tab. The Forecast Chart is the default visualization.
Note: You might experience errors with forecast charts that exceed 50 thousand points (input + output).
The following images shows an example Forecast Chart:

This image shows an example Forecast Chart visualization with sample data.

All parameters have default settings that can be modified.

Along the bottom of the chart you can view and scroll a signal's history. Signal history lets you observe patterns in the time series. Drag the control to quickly scroll to sections of the time series' history. Use the handles on the control to increase or decrease the zoom level.

On the chart you can zoom in and click to see more details on the forecast as shown in the following image:

This image shows a zoomed in view of the same Forecast Chart. Hovering over the chart displays more information about data points and points in time.

Anomaly Detection syntax and parameters

Follow this syntax to perform anomaly detection on the forecast:

mode=anomaly

You add mode=anomaly after | apply CDTSM as shown in the following example:

CODE 
| inputlookup internet_traffic.csv 
| apply CDTSM bits_transferred mode=anomaly

You can use the following parameters for anomaly detection on your CDTSM forecasts. You can choose to keep or adjust the default values:

Note: Default values are in bold
Parameter Values Description
mode

anomaly, forecast

 Sets which functionality is invoked, forecasting or anomaly detection.
conf_interval 20, 40, 50, 60, 80, 90, 98,99

forecast: 80

anomaly: 60

 This is the confidence band, in percentage, around the forecasted value.
method

quantile, iqr_residual

 The algorithm used for determining if a datapoint is anomalous. For more information see the Anomaly Detection method comparison.
quantile_upper

p60, p70, p75, p80, p90, p95, p99

 Quantile for threshold calculation above the forecasted mean​.
quantile_lower

p1, p5, p10, p20, p25, p30, p40

 Quantile for threshold calculation below the forecasted mean​.
multiplier Any float value.

quantile: 5

iqr_threshold: 3

 Sensitivity multiplier to determine the anomaly threshold. For method: quantile, threshold is the (quantile-mean) x sensitivity threshold. Similar effect for method: iqr_residual
threshold_direction Both, upper, lower Direction(s) to evaluate for anomalies. Above or below the forecasted mean
on_span Any float value. Default 3. The number of data points evaluated in a sliding window when determining if an anomaly segment should start or end.
off_span Any float value. Default 3. The number of data points evaluated in a sliding window when determining if an anomaly segment should start or end.
on_ratio Decimal value.

>0, <= 1

on_ratio: 0.5

 The percentage of data points in the on_span and off_span windows that must be outside of the threshold band in order to start or end an anomaly segment. An anomaly starts at the first anomalous data point.​
off_ratio Decimal value.

>0, <= 1

off_ratio: 1.0

 The percentage of data points in the on_span and off_span windows that must be outside of the threshold band in order to start or end an anomaly segment. An anomaly starts at the first anomalous data point.​
detection_window_earliest Relative or absolute time. Any Splunk supported date / time format. Default: -3h The start of the anomaly detection window
detection_window_latest Relative or absolute time. Any Splunk supported date / time format. Default: -3h The end of the anomaly detection window. If empty, continue until the end of the time series. ​

Anomaly Detection syntax constraints

Do not use the following forecasting parameters when adding mode=anomaly to the Cisco Deep Time Series Model. Use these for forecasting only, not forecasting with anomaly detection:

  • forecast_k

  • holdback

  • quantiles

Anomaly Detection method comparison

The Cisco Deep Time Series Model produces a probabilistic forecast. Not just a mean, but a full range of quantiles. Anomaly detection reuses that forecast to answer one question: Is this observation surprising enough to flag?

The method parameter is the algorithm used for determining if a datapoint is anomalous. See the following table for a comparison of the 2 method parameter options:

Method Question Syntax When to use this method
Quantile Is the value outside the forecast's confidence band? method=quantile Use quantile when you have a single series, clearly-shaped signals; easy to explain to customers:

  • Compares the actual value directly to TSFM quantiles (default p20–p80).​

  • Threshold band breathes with forecast uncertainty​.

  • Sensitivity multiplier widens the band to reduce false positives.​

  • Very intuitive. Reads like a confidence-interval check.
IQR Residual How big is the error, compared to the series' typical spread?​ method=iqr_residual Use IQR when you have fleets of diverse metrics where per-series tuning is impractical:

  • Scores each point: |actual − forecast mean| ÷ running_mean(IQR).​

  • Normalizes by the series' own variability : scale-invariant.​

  • One threshold works across very different metrics.

Using Anomaly Detection visualizations

When you add or include mode=anomaly to the call to the CDTSM, you can view the results using the Anomaly Detection Chart visualization option on the Visualization tab as shown in the following image:

This image shows an example search in the AI Toolkit that uses the CDTSM for forecasting and anomaly detection.

All parameters have default settings that can be modified. The default is to detect anomalies in the last 3 hours.

Along the bottom of the chart you can view and scroll a signal history. Signal history lets you observe patterns in the time series. Drag the control to quickly scroll to sections of the time series' history. Use the handles on the control to increase or decrease the zoom level.

On the chart you can zoom in and click to see more details on the forecast as shown in the following image:

Note: The default confidence intervalis between the 80th and 20th quantiles. This is referred to as the 60% confidence interval

This image shows an example search in the AI Toolkit that uses the CDTSM for forecasting and anomaly detection. This is a more zoomed in view showing how you can see more details when you hover over points on the Anomaly Detection Chart visualization.

Anomaly boundaries shown by red bands​. Anomaly segments are indicated with red rectangles​ The anomaly threshold is the upper and lower red bands. The threshold value is the quantile x , or what you can call a "sensitivity multiplier:. The sensitivity multiplier is the main control for adjusting for false alerts.

Cisco Deep Time Series Model behavior

The model expects data with a fixed resolution. For example1 minute, 5 minutes, or 10 minutes.

CAUTION: If you pass in data that is not fixed resolution the command will error out.
The model returns mean forecast and lower5 and upper95 as per the default confidence interval90%.

When the model returns mean forecast it is labeled as predicted.

The model returns quantiles information only until reaching 128 forecast_k data points.After that the quantiles information might not be present. If forecast_k is greater than 384, expect some degraded predictions.

You can forecast more than 1 value by placing those fields after the apply CDTSM commands. For example, to forecast 2 columns such as cpu and memory, the ML-SPL search is apply CDTSM cpu memory.

The model requires an input of at least 60 data points.

The time_field parameter is not a required parameter. If not passed, _time is used as the time_field by default. The algorithm expects every value of the time_field to be populated although the data might have a fixed resolution.
CAUTION: To prevent an error the time_field must be populated with date values.
You can exclude both the holdback and forecast_k parameters. The default value of forecast_k is 128, and the default value of holdback is 0. If these parameters are included in the ML-SPL search they cannot both be 0.

When you pass in both holdback and forecast_k you must account for the holdback parameter in the forecast_k parameter. For example, if you want 10 new timestamps and you require 5 input points to be considered as holdback, then set holdback as 5 and forecast_k as 10 + 5 = 15. Meaning holdback is always less than or equal to forecast_k.

The quantiles parameter is not mandatory. If passed in the value must be a single percentile value like p10 or p20, or multiple percentile values in a comma separated value (CSV) format. Permissible quantile values are mean, p1, p10, p20, p25, p30, p40, p5, p50, p60, p70, p75, p80, p90, p95, and p99.

Confidence interval is mapped to quantiles. For example conf_interval 90 means q0.5 lower bound, and q0.95 upper bound [ lower = 1 - (90/100) / 2 = 0.05, and upper=1 - ( 1 -(90/100) / 2) = 0.95 ].

The CDTSM only supports specific quantiles and in turn supports only the confidence intervals of 20, 40, 50, 60, 80, 90, or 98.

Troubleshooting the CDTSM

I opted out when I saw the option to try the CDTSM preview. How can I opt back in?

There is no option in the AI Toolkit itself to opt back in to try the CDTSM preview. Contact Splunk support to get this preview made available.

Specifying multiple quantiles does not seem to work?

You must apply quotes when passing in multiple, comma separated values to a parameter.