Latency in Ingest monitoring

Latency measures the time elapsed between when an event is generated and when it is indexed in Splunk. It is calculated as the difference between the event's timestamp and its index time.

For example, if an event has a timestamp of 2:01:00 PM and is indexed at 2:01:30 PM, the latency is 30 seconds.

"Latency Index Time" is the index time of the last event ingested for a data entity.

How latency is measured

Ingest monitoring performs searches every 5 minutes to monitor ingested events. These searches evaluate events with timestamps ranging from 7 days in the past to 24 hours in the future, supporting a latency range of -7 days to +24 hours.

To ensure efficient and accurate latency calculations, Ingest monitoring does the following:

1. Filters results by indexing events indexed within a 5-minute range (from 10 minutes ago to 5 minutes ago).
2. Groups events by index, source type, host, source, and 5-minute time periods.
3. For each group, latency is calculated as the difference between the latest index timestamp and the latest event timestamp.

This process improves accuracy and avoids the resource-intensive process of calculating latency for every individual event.

The calculated latency, along with the corresponding index, source, source type, host, and event count, is saved in the summary index for reporting and analysis.

Negative latency

In some cases, Ingest monitoring may report negative latency. This occurs when the index timestamp precedes the event timestamp, which often indicates an issue with timestamp extraction. Negative latency typically points to a problem in how event timestamps are parsed or processed.