Normalize alerts with correlation search templates in ITSI

IT Service Intelligence (ITSI) ships with several predefined correlation search templates to help you normalize alerts from common third-party systems. Leverage these searches when creating a correlation search to bring third-party alerts into ITSI and normalize them as notable events. For more information about correlation searches, see Overview of correlation searches in ITSI.

Prerequisites

Requirement Description
ITSI role You must have the write_itsi_correlation_search capability to create a correlation search. The itoa_admin and itoa_team_admin ITSI roles have this capabilities by default.
Ingest third-party data You must be ingesting data from the corresponding third-party alerting system into Splunk Enterprise in order to normalize it in ITSI. Optionally, you can install the related Splunk add-on for that system. The table below lists the add-ons related to each search, if available.

Access correlation search templates

All third-party search templates are available within the correlation search creation workflow. To leverage a template, perform the following steps:

  1. From the ITSI main menu, click Configuration > Correlation Searches.
  2. Click Create New Search > Create Correlation Search.
  3. Provide a name and description for the search.
  4. For Search Type, choose Predefined.
  5. Click Select a Search and choose from one of the predefined search templates described below.
  6. Click Select an index and choose an index to use for the search.
  7. Configure the rest of the correlation search to normalize the third-party alert fields. For instructions, see Ingest third-party alerts into ITSI.

Available correlation search templates

Choose from the following correlation search templates to bring third-party alerts into ITSI:

Search name Search Description
BMC TrueSight Events 
index=* | eval itsi_host=alias_host | eval tmp_entity=alias_host | eval itsi_eventtype=alias_parameter | eval itsi_class=CLASS | eval itsi_message=Msg | eval itsi_incident=itsm_incident_id | eval itsi_ip=mc_host_address | eval itsi_support=support_group | eval itsi_severity=case(mc_original_severity="CRITICAL", 6, mc_original_severity="OK", 2, 1=1,1) | dedup alias_host alias_parameter Msg
 BMC Truesight (patrol, msend) stateful events. Deduplicated by alias_host, alias_parameter, Msg.
MuleSoft Events 
index=* sourcetype=httpevent severity!=INFO | dedup source | eval tmp_entity=source | eval itsi_severity=case(severity=="WARN", 3, severity=="ERROR", 5, 1=1,1) | eval itsi_message=message | eval itsi_eventtype=logger
 MuleSoft stateful related events, filtering out severity=INFO, deduplicated by source.
Nagios Events 
index=* sourcetype=nagiosserviceperf | eval norm_severity=case(severity=="CRITICAL",6,severity=="WARNING",4, severity="OK", 2) |  dedup consecutive=true src_host severity name | eval tmp_entity=host | eval host=src_host
 Nagios stateful performance events. Filtering by sourcetype=nagiosserviceperf, deduplicated by consecutive, src_host, severity, name.

Add-on: Splunk Add-on for Nagios Core
Netcool Events 
index=netcool | eval itsi_host=NODE | eval itsi_eventID=ALERTID | eval itsi_alertKey = ALERTKEY | dedup consecutive=true itsi_host itsi_alertID itsi_alertKey | eval itsi_tally = TALLY | eval itsi_orig_severity = SEVERITY | eval itsi_severity = case(SEVERITY==0,2, SEVERITY==1,2, SEVERITY==2,3,SEVERITY==3, 4, SEVERITY==4, 5, SEVERITY==5, 6, 1=1, 1) | eval itsi_message = SUMMARY | eval tmp_entity = APPLICATION | eval itsi_length=len(_raw) | eval itsi_identifier = IDENTIFIER + LASTOCCURRENCE  + STATECHANGE | eval itsi_time=strftime(LASTOCCURRENCE,"%F %T")
 Netcool stateful performance events. Deduplicated by consecutive, itsi_host , itsi_alertID , itsi_alertKey.
NewRelic Events 
index=* sourcetype=newrelic* | eval tmp_entity=norm_instance | eval itsi_severity=case(health_status=="green", 2, health_status=="red", 6)  | search itsi_severity!=2 | dedup transaction_name health_status
 New Relic stateful events. Filtering by sourcetype=newrelic*, deduplicated by transaction_name, health_status.

Add-on: Splunk Add-on for New Relic
ScienceLogic em7 
index=* | dedup em7_var_evententityname em7_var_alertid | eval itsi_category = em7_var_categoryname | eval itsi_message = em7_var_eventmessage | eval itsi_orig_severity = em7_var_eventseveritytext | eval itsi_host = em7_var_evententityname | eval tmp_entity = em7_var_evententityname | eval itsi_url = em7_var_eventurllink | eval itsi_ip = em7_var_ipaddress | eval itsi_location = em7_var_slsystemname | eval itsi_class = class | eval itsi_eventID = em7_var_alertid | eval itsi_supportGroup = em7_var_support_group | eval itsi_backLink = em7_var_device_back_link | eval itsi_severity=case(em7_var_eventseveritytext=="NOTICE", 3, em7_var_eventseveritytext=="MINOR", 4, em7_var_eventseveritytext=="MAJOR", 5, em7_var_eventseveritytext=="CRITICAL", 6, 1=1,1)
 ScienceLogic em7 stateful events. Deduplicated by em7_var_evententityname, em7_var_alertid (used by notable event identifier fields).
SolarWinds Events 
index=* | eval itsi_host=NodeName | eval tmp_entity=NodeName | eval itsi_orig_severity=Severity | eval itsi_severity=Severity | eval itsi_alert_type=eventtype | eval itsi_NodeDescription=NodeDescription | eval itsi_vendor=Vendor | eval itsi_summary=StatusDescription | eval itsi_category=tag | eval itsi_location=Location | eval itsi_severity=case(Severity<=500, 2, Severity<=4000, 3, Severity<=9000, 4, Severity<=12000, 5 , Severity>=15000, 6, 1=1, 1) | dedup NodeName eventtype StatusDescription
 SolarWinds stateful events, not performance metrics. Deduplicated by NodeName, eventtype, StatusDescription.

Add-on: SolarWinds Add-on for Splunk
Unix or Linux Events 
index=* status=Stopped | dedup host status Description | eval itsi_severity=status | eval itsi_host=host | eval itsi_eventtype=eventtype | eval tmp_entity=host | eval itsi_severity=case(status=="Stopped", 4, 1=1,1)
 Unix and Linux-based stateful events using the field Status as severity. If clearing events (Up) are being ingested, remove the filter for status=Stopped (clearing events can be used to automatically clear notable events). Deduplicated by host, status, and Description.

Add-on: Splunk Add-on for Unix and Linux
WinEvent:System or WinEvent:Application 
index=* sourcetype=WinEventLog severity!=informational | rename event_id AS orig_event_id | eval tmp_entity=host |eval itsi_eventtype=eventtype | eval itsi_message=Message | eval itsi_ip=Source_Network_Address | eval itsi_fqdn=ComputerName | eval itsi_event_category=category | eval itsi_severity=case(severity=="medium", 4, severity=="high", 5, 1=1,1) | dedup host Message orig_event_id
 Windows-based stateful events from winevents:system and winevents:application. Filtering out informational events and deduplicated on Message, host, and orig_event_id.

Add-on: Splunk Add-on for Microsoft Windows

AppDynamics

Search name Search Description
Events 
index=* source=application_events | eval itsi_application=application_name | eval itsi_customer_name=account_name | eval itsi_application_id=application_id | rename application_events{}.severity as itsi_raw_severity | spath output=itsi_affectedEntities path=application_events{}.affectedEntities{}.name  | spath output=itsi_triggeredEntity path=application_events{}.triggeredEntity.name | spath output=itsi_subType path=application_events{}.subType | spath output=itsi_summary path=application_events{}.summary | eval itsi_severity=if(itsi_raw_severity=="MINOR","4",if(itsi_raw_severity=="WARN","3",if(itsi_raw_severity=="NORMAL","2",if(itsi_raw_severity=="ERROR","5","6")))) | dedup itsi_triggeredEntity itsi_application itsi_subType
 AppDynamics stateful events based on the ingest events from AppDynamics, using spath to expand the key-value pairs into single fields. Deduplicated by itsi_triggeredEntity, itsi_application, and itsi_subType.

Add-on: Splunk Add-on for AppDynamics
Health Rule Violations 
index=* source=healthrule_violations | dedup healthrule_violations{}.affectedEntityDefinition.entityId healthrule_violations{}.deepLinkUrl | spath output=itsi_ruleViolation path=healthrule_violations{}.name  | spath output=itsi_affectedEntityDefinition path=healthrule_violations{}.affectedEntityDefinition.name | eval tmp_entity=itsi_affectedEntityDefinition | spath output=event_description path=healthrule_violations{}.description | spath output=severity path=healthrule_violations{}.severity | spath output=itsi_triggered_entity_name path=healthrule_violations{}.triggeredEntityDefinition.name | spath  output=itsi_entity_type path=healthrule_violations{}.triggeredEntityDefinition.entityType | eval eventsource="APPD" | eval svc_entity=itsi_affectedEntityDefinition | eval itsi_severity=case(severity == "CRITICAL", 5,severity == "WARNING", 3) | eval tmp_entity=svc_entity
 AppDynamics health rule stateful violations based on the ingest of health rule events from AppDynamics, using spath to expand the key-value pairs into single fields. Deduplicated on healthrule_violations{}.affectedEntityDefinition.entityId and healthrule_violations{}.deepLinkUrl.

Add-on: Splunk Add-on for AppDynamics