Guidelines to create a custom finding-based detections

Note: Finding-based detections in Splunk Enterprise Security are currently released as a preview feature. Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms. To provide feedback, visit the Voice of the Customer portal for finding-based detections and select Send Feedback.

Splunk Enterprise Security uses risk-based alerting so that finding-based detections can provide high-confidence, aggregated alerts for investigations. To leverage risk-based alerting, you must customize finding-based detections as follows:

Include macros in the finding-based detection

The SPL of the custom finding-based detection must include one of the following macros with 1 to 5 arguments:

  • `fbd_grouping(1)`
  • `fbd_grouping(2)`
  • `fbd_grouping(3)`
  • `fbd_grouping(4)`
  • `fbd_grouping(5)`

These macros set the value of the `fbd_group_by` field, which is a part of the SPL search results if the `common_fbd_fields_results` macro is also used. If you don't use the `fbd_grouping` macro, you must include a new `fbd_group_by` field in the SPL search results of your custom finding-based detection. If you don't use the 'fbd_group_by' field, multiple unrelated entries are grouped into one finding group, which might impact the display of the finding groups in the analyst queue.

Following is an example of the SPL for a custom finding-based detection that includes the fbd_grouping and the common_fbd_fields_results macros: 

| tstats `summariesonly` `common_fbd_fields`, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, values(source) as contributing_source, values(All_Risk.cim_entity_zone) as cim_entity_zone from datamodel=Risk.All_Risk where 
    [| generatetimerange "Threat - ATT&CK Tactic Threshold Exceeded For Object Over Previous 7 Days - Rule" 
| return earliest, latest ] All_Risk.annotations.mitre_attack.mitre_tactic_id=* by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index 
| rename All_Risk.risk_object_type as risk_object_type, All_Risk.normalized_risk_object as normalized_risk_object, annotations.mitre_attack.mitre_tactic_id as mitre_tactic_id1, annotations.mitre_attack.mitre_technique_id as mitre_technique_id1 
| `generate_findings_summary` 
| stats list(*) as * limit=1000, sum(int_risk_score_sum) as risk_score by `fbd_grouping(normalized_risk_object, risk_object_type)` 
| `dedup_and_compute_common_fbd_fields`, contributing_source=mvdedup(contributing_source), mitre_tactic_id1=mvdedup(mitre_tactic_id1), mitre_technique_id1=mvdedup(mitre_technique_id1), contributing_source_count=mvcount(contributing_source), mitre_tactic_id_count=mvcount(mitre_tactic_id1), mitre_technique_id_count=mvcount(mitre_technique_id1), threat_object=mvdedup(threat_object), cim_entity_zone=mvdedup(cim_entity_zone) 
| fillnull value=0 mitre_tactic_id_count, mitre_technique_id_count 
| rename mitre_tactic_id1 as annotations.mitre_attack.mitre_tactic_id, mitre_technique_id1 as annotations.mitre_attack.mitre_technique_id 
| eval annotations.mitre_attack='annotations.mitre_attack.mitre_technique_id' 
| fields - int_risk_score_sum, int_findings_count, individual_threat_object_count, contributing_event_ids 
| `drop_dm_object_name("All_Risk")` 
| where mitre_tactic_id_count >= 3 and contributing_source_count >= 4 
| table `common_fbd_fields_results`, annotations.mitre_attack.mitre_tactic_id, mitre_tactic_id_count, annotations.mitre_attack.mitre_technique_id, mitre_technique_id_count, cim_entity_zone, contributing_source, contributing_source_count, annotations.mitre_attack

Following is an example of an SPL for a custom finding-based detection without the fbd_grouping macro:

Note: The fbd_group_by must be included in the SPL.
| tstats `summariesonly` `common_fbd_fields`, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, values(source) as contributing_source, values(All_Risk.cim_entity_zone) as cim_entity_zone from datamodel=Risk.All_Risk where 
    [| generatetimerange "Threat - A Different Custom FBD Detection - Rule" 
| return earliest, latest ] All_Risk.annotations.mitre_attack.mitre_tactic_id=* by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index 
| rename All_Risk.risk_object_type as risk_object_type, All_Risk.normalized_risk_object as normalized_risk_object, annotations.mitre_attack.mitre_tactic_id as mitre_tactic_id1, annotations.mitre_attack.mitre_technique_id as mitre_technique_id1 
| `generate_findings_summary` 
| stats list(*) as * limit=1000, sum(int_risk_score_sum) as risk_score by normalized_risk_object, risk_object_type, abc, def, ghi, jkl, mno, pqr 
| eval fbd_group_by=normalized_risk_object."-".risk_object_type."-".abc."-".def."-".ghi."-".jkl."-".mno."-".pqr 
| `dedup_and_compute_common_fbd_fields`, contributing_source=mvdedup(contributing_source), mitre_tactic_id1=mvdedup(mitre_tactic_id1), mitre_technique_id1=mvdedup(mitre_technique_id1), contributing_source_count=mvcount(contributing_source), mitre_tactic_id_count=mvcount(mitre_tactic_id1), mitre_technique_id_count=mvcount(mitre_technique_id1), threat_object=mvdedup(threat_object), cim_entity_zone=mvdedup(cim_entity_zone) 
| fillnull value=0 mitre_tactic_id_count, mitre_technique_id_count 
| rename mitre_tactic_id1 as annotations.mitre_attack.mitre_tactic_id, mitre_technique_id1 as annotations.mitre_attack.mitre_technique_id 
| eval annotations.mitre_attack='annotations.mitre_attack.mitre_technique_id' 
| fields - int_risk_score_sum, int_findings_count, individual_threat_object_count, contributing_event_ids 
| `drop_dm_object_name("All_Risk")` 
| where mitre_tactic_id_count >= 3 and contributing_source_count >= 4 
| table `common_fbd_fields_results`, annotations.mitre_attack.mitre_tactic_id, mitre_tactic_id_count, annotations.m

If the `dedup_and_compute_common_fbd_fields` macro is not included in the SPL for your custom finding-based detection, you must add is_finding_group="True" to the SPL of the custom detection in order for it to generate a finding group.

Do the following if the `dedup_and_compute_common_fbd_fields` macro is not included in the SPL:

  • Add is_finding_group to the table statement.
  • Set is_finding_group to True after an evaluation statement.

For example: 

eval is_finding_group="True" ...
table `common_fbd_fields_results`, is_finding_group, ...

Include the Risk data model in the finding-based detection

Note: New fields are added to the Risk data model to create finding groups. After upgrading to Splunk Enterprise Security version 8.0, the new Risk data model overwrites all customizations that you might have made. Contact Splunk support within 90 days of an upgrade to retrieve any old configurations.

All finding-based detections must have the following base search, which includes the Risk data model: 

| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone from datamodel=Risk.All_Risk

Include the generatetimerange command in the the finding-based detection

Use the generatetimerange command to provide an absolute time range for each detection. When the SPL search of the detection runs, generatetimerange command checks that the detection name provided is valid and that the time range associated with the detection is within range of the specified max append time.

If the current time comes after the latest time field, the earliest time and latest time fields are overwritten by the new time range generated by the generatetimerange command.

The name of the detection must be wrapped in quotation marks after the generatetimerange keyword. Additionally, a return must be stated for the earliest time and the latest time fields after the generatetimerange command.

The shell of the command reads as follows:

[ | generatetimerange "Detection Name" | return earliest, latest ]

To add this command to your custom detection SPL, add the command to a where clause after the Risk datamodel is defined in the search.

The search reads as follows: 

| tstats … from datamodel=Risk.All_Risk where [ | generatetimerange "detection name" | return earliest, latest ]

The generatetimerange command is similar to using the earliest time and the latest time fields in a search as follows: 

| spl search where earliest="...", latest="..." | ...

Include specific fields in the search to group findings

Additional fields can be added to the SPL search for the finding-based detection based on specific requirements. Adding fields in the SPL search helps to customize the display of findings in the analyst queue, the Risk Timeline, and the investigations

For example, the following SPL search groups findings by fields such as cim_entity_zone: 

| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10

The following search adds the generatetimerange command and groups findings by cim_entity_zone in the SPL search: 

| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where [ | generatetimerange "Detection Name" | return earliest, latest ] where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10