Guidelines to create a custom finding-based detection

Splunk Enterprise Security uses risk-based alerting so that finding-based detections can provide high-confidence, aggregated alerts for investigations. To leverage risk-based alerting, you must customize finding-based detections as follows:

Edit detection SPL templates and macros for finding-based detections

You can edit these templates to create finding-based detections based on your specific requirements.

Detection SPL templates

The following detection SPL templates are included with Splunk Enterprise Security as example queries to help understand the logic behind the detection editor. You can edit these templates to create finding-based detections based on your specific requirements.

  • Finding Group Template - Entity Exceeded MITRE ATT&CK Tactic Threshold

  • Finding Group Template - Entity Exceeded MITRE ATT&CK Technique Threshold

  • Finding Group Template - Entity Exceeded Risk Threshold

  • ESCU - Finding Group - Entity Exceeded Threshold with Multiple Findings

  • Finding Group Template - Entity Exceeded Kill Chain Threshold

Macros to create finding-based detections

Use macros to simplify the process of creating SPL search queries for finding-based detections by standardizing the aggregation of findings and entity-level grouping. Using these macros helps to ensure consistency and maintainability of the SPL query structure​ for finding-based detections.

Macro Description SPL query
generate_findings_summary Differentiates and extracts finding data from notable and risk indexes.​ eval count_findings = if(index == "notable", int_findings_count, 0), finding_ids=if(index == "notable", contributing_event_ids, NIL), count_intermediate_findings = if(index == "risk", int_findings_count, 0), intermediate_finding_ids=if(index == "risk", contributing_event_ids, NIL), intermediate_findings_summary = if(index == "risk", json_object("risk_object", normalized_risk_object, "risk_object_type", risk_object_type, "count", int_findings_count, "risk_score", int_risk_score_sum), NIL), is_finding_group="True"
calculate_findings_fields Aggregates and enriches finding statistics​ definition = stats sum(int_risk_score_sum) as risk_score, dc("annotations.analytic_story") as analytic_story_count, dc("annotations.mitre_attack.mitre_tactic") as mitre_tactic_id_count, dc("annotations.mitre_attack") as mitre_technique_id_count, dc("annotations.kill_chain_phases") as kill_chain_phases_count, dc("annotations.nist") as nist_count, dc("annotations.cis20") as cis20_count, dc(risk_message) as risk_message_count, dc(threat_object_info) as threat_object_count, dc(orig_source) as source_count, values(*) as *, sum(count_findings) as count_findings, sum(count_intermediate_findings) as count_intermediate_findings, sum(eval(count_intermediate_findings+count_findings)) as total_event_count​
generate_findings_summary_on_entity_for_drilldown_searches Creates end-to-end entity-level finding group based on drill-down searches definition = `generate_findings_summary` | `calculate_findings_fields` by `fbd_grouping(normalized_risk_object, risk_object_type)` | eval finding_ids = mvindex(finding_ids, 0, 99) , intermediate_finding_ids = mvindex(intermediate_finding_ids, 0, 99), risk_event_count = count_intermediate_findings | fields - int_risk_score_sum, int_findings_count, contributing_event_ids​

Following is an example of the SPL for a custom finding-based detection that includes thegenerate_time_range macro:

PYTHON 
| tstats `summariesonly` `common_fbd_fields` from datamodel=Risk.All_Risk where [| `generate_time_range("Threat - Finding Group Template - Entity Exceeded Risk Threshold - Rule")` | return earliest, latest ] by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index | `drop_dm_object_name("All_Risk")` | `get_mitre_annotations` | `generate_findings_summary_on_entity` | where risk_score > 100
```

Changes in detection macros in Splunk Enterprise Security version 8.4

In Splunk Enterprise Security version 8.4, the following fields are added to the common_fbd_fields macro in addition to several annotation-related fields:

  • values(All_Risk.threat_object) as threat_object
  • values(All_Risk.src_user) as src_user
  • values(All_Risk.risk_message) as risk_message

In Splunk Enterprise Security version 8.2, these fields were included outside the macro in the finding-based detection searches.

Note: If you created custom finding-based detection searches in Splunk Enterprise Security version 8.2 using these macros, you might see duplication field definition errors.

For example, if you created a custom finding-based detection using the default macros available in Splunk Enterprise Security version 8.2 similar to the following: | tstats `summariesonly` `common_fbd_fields`, values(All_Risk.threat_object) as threat_object from datamodel=Risk.All_Risk...

Then, in Splunk Enterprise Security version 8.4, the threat_object field can be retrieved twice, once inside the macro and once independently on its own, which leads to a duplicate field definition error when invoking the tstats command.

If this occurs, you must delete the values of (All_Risk.threat_object) for the threat_object field from the detection searches since it's now included in the macro.

You don't need to take any action if you used the finding-based detections that were available by default in Splunk Enterprise Security version 8.2 without any modifications. All default detections available in Splunk Enterprise Security are updated. Custom detections don't use the common_fbd_fields macro.

Cloned finding-based detections point to incorrect detection name

When cloning an existing finding-based detection, the generate_time_range macro in the cloned detection points to the original detection name instead of the cloned detection name. When a detection is cloned, the detection name parameter inside the search SPL is not automatically updated to reflect the new detection name.

If you clone a finding-based detection, you must update the detection name using the following method:

  1. Update the SPL search by identifying the generate_time_range macro in the SPL after you have cloned the detection and then update the detection name prior to saving the detection. For example, in the following SPL search | generate_time_range("Original Detection Name") | return earliest, latest], replace the generate_time_range macro as follows: | generate_time_range("Your Cloned Detection Name") | return earliest, latest]
  2. Review any configured drilldown searches and update references to the detection name, if available. 
  3. After the detection runs, verify the detection creates its own entry in the lookup by running the following command:  | inputlookup detection_time_range_lookup | search detection_name="Your New Detection Name"  When you run the command, a unique entry for your cloned detection is displayed.

Include the Risk data model in the finding-based detection

Note: New fields are added to the Risk data model to create finding groups. After upgrading to Splunk Enterprise Security version 8.0, the new Risk data model overwrites all customizations that you might have made. Contact Splunk support within 90 days of an upgrade to retrieve any old configurations.

All finding-based detections must have the following base search, which includes the Risk data model:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone from datamodel=Risk.All_Risk

Include specific fields in the search to group findings

Additional fields can be added to the SPL search for the finding-based detection based on specific requirements. Adding fields in the SPL search helps to customize the display of findings in the analyst queue, the Risk Timeline, and the investigations

For example, the following SPL search groups findings by fields such as cim_entity_zone:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10

The following search adds the generate_time_range(<fbd_detection_name> macro and groups findings by cim_entity_zone in the SPL search:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where [ | generate_time_range(<fbd_detection_name> | return earliest, latest ] where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10