Static Threshold

The Static Threshold condition alerts when a signal goes above or below a static threshold, or is within or outside of a range.

Static Threshold alerts when a signal goes above or below a static threshold, or is within or outside of a range. Use this condition when you need to be alerted based on fixed values, as opposed to trends (for which you can use the Sudden Change and Resource Running Out conditions) or comparisons with past behavior (for which you can use Historical Anomaly). This condition works best with metrics that have a static range of "good" and "bad" values.

To compare one signal with another, use Custom threshold.

Examples

  • You have an availability SLA of 99.9, and want to be alerted whenever your availability drops below that value.

  • You have a latency signal with a "healthy" range between 200 and 300 ms, and want to be alerted when it falls outside that range.

Suggested Threshold

The Suggested Threshold feature provides a recommended value based on the historical performance of your chosen metric. By selecting this option, you can set accurate alert conditions automatically, removing the need to manually analyze past signal behavior, and reduce the risk of setting thresholds that are too low (causing alert noise) or too high (missing real incidents).

The suggested value is specific to the signal currently defined in the detector. If you change the scope of the signal—such as filtering a specific service, environment, or host—select Suggested Threshold again to recalculate the value for the new data set. In addition, the display unit of the value is based on what was chosen for the detector and will adjust accordingly. For example, if your signal is configured to display in milliseconds, the suggested value is provided in milliseconds.

Threshold calculation

The threshold is based on the median absolute deviation (MAD) from the last 14 days. The calculation accounts for normal variability and ensures alerts are only triggered for significant deviations. Using the median instead of the mean provides more stable and reliable thresholds that are resistant to outliers and less susceptible to overcorrection.

Settings

Parameter

Values

Notes

Alert when

Above, Below, Out of Range, Within Range

Threshold

Lower threshold, Upper threshold

Number

Enter a fixed value or select Suggested Threshold to calculate a value based on historical data.

Lower and upper thresholds are available if you choose Out of Range or Within Range for Alert when.

For Alert when, if you select Out of Range or Within Range, you can enter a number for Lower threshold and Upper threshold.

For Within Range, the threshold values you provide are inclusive ( >= and <= ).

For Out of Range, the threshold values you provide are exclusive ( > and < ).

Trigger sensitivity

Immediately, Duration, Percent of duration

The default is 80% of 5 minutes (Percent of duration) to reduce alert noise.

Immediately triggers an alert as soon as the threshold is met.

Duration triggers when the signal meets and remains at threshold condition for a specified period, such as 10 minutes. If it is normal for a signal to rise and fall rapidly, using this option reduces flappiness. If this option is used, an alert will not trigger if there are any missing data points during the duration. For more information, see Duration to trigger an alert.

Percent of duration triggers based on the number of data points that met the threshold during the specified duration. For more information, see Duration to trigger an alert.

Duration

Integer >= 1, followed by time indicator (s, m, h, d, w). For example, 30s, 10m, 2h, 5d, 1w.

The amount of time the signal must meet the threshold condition. Longer time periods result in lower sensitivity and potentially fewer alerts.

Percent of duration

Percentage: Integer between 1 and 100; Duration: Integer >= 1, followed by time indicator (s, m, h, d, w). For example, 30s, 10m, 2h, 5d, 1w.

The percentage of anomalous data points received during the specified duration.

Duration to trigger an alert

As you might expect, choosing Immediately for Trigger Sensitivity means that an alert will be triggered as soon as the signal meets the threshold. This option is the most sensitive (might trigger the most alerts) of the 3 trigger sensitivity options.

Depending on the nature of your signal, triggering alerts immediately can lead to flappiness. In these cases, you can choose one of the other options, Duration or Percent of duration.
Note: To ensure alerts represent sustained issues rather than temporary noise, the 80% of 5 minutes baseline is recommended. While you can change this to Immediately for critical high-priority signals, using a percentage of duration ensures that transient spikes do not trigger unnecessary notifications.
The Duration option triggers when the signal meets and remains at threshold condition for a specified period, such as 10 minutes. Therefore, using this option is less sensitive (might trigger fewer alerts) than the Immediately option. If you use this option, an alert isn’t triggered if any data points are delayed or don’t arrive at all during that time range, even if all the data points that are received do meet the threshold. For more information about delayed or missing data points, see Handle delayed or missing data points.

If you want an option that triggers even if some data points do not arrive on time, use Percent of duration (with a percentage below 100).

The Percent of duration option triggers alerts based on the number of data points that met the threshold during the window, compared to how many data points were expected to arrive. Because this option triggers an alert based on the percentage of data points that met the threshold, it can sometimes trigger an alert even if some data points didn’t arrive on time. Therefore, using this option with a percentage below 100 is more sensitive (might trigger more alerts) than the Duration option.

The following examples illustrate how alerts are triggered in various situations.

Example 1

  • Option you specify for Trigger Sensitivity: Duration = 3 minutes

  • Resolution of the signal: 5 seconds

  • Number of data points expected in 3 minutes: 12 per minute * 3 minutes (36)

  • Number of anomalous data points (how many times the threshold must be met) to trigger alert: 36

    Total data points expected

    Total data points received

    Anomalous data points required

    Anomalous data points received

    Alert is triggered?

    36

    36

    36

    36

    Yes

    36

    36

    36

    35 or fewer

    No

    36

    35

    36

    35 or fewer

    No

Example 2

  • Option you specify for Trigger Sensitivity: Percent of Duration = 75% of 3 minutes

  • Resolution of the signal: 5 seconds

  • Number of data points expected in 3 minutes: 12 per minute * 3 minutes (36)

  • Number of anomalous data points (how many times the threshold must be met) to trigger alert: 75% of 36 (27)

    Total data points expected

    Total data points received

    Anomalous data points required

    Anomalous data points received

    Alert is triggered?

    36

    36

    27

    27-36

    Yes

    36

    30

    27

    27-30

    Yes

    36

    30

    27

    26 or fewer

    No

    Note that in the last example above, even if 26 anomalous data points arrive, and 26/30 is greater than the 75% you specified, the required number of anomalous data points (27) did not arrive. Therefore, the alert isn’t triggered. The percent you specify represents percent of expected data points, not percent of received data points.

API usage note

If you use the Splunk Observability Cloud API to build detectors, you can use the const() function to specify different threshold values for different dimension values. For example, you might have different acceptable SLA values depending on whether a host is in dev, lab, or production. Using the const() function can be more efficient than manually building multiple detectors or detectors with multiple rules. For more information, see the section on creating multiple time series in the Splunk Observability Cloud API documentation .