Guidelines to create a custom finding-based detection

Splunk Enterprise Security uses risk-based alerting so that finding-based detections can provide high-confidence, aggregated alerts for investigations. To leverage risk-based alerting, you must customize finding-based detections as follows:

Edit detection SPL templates and macros for finding-based detections

You can edit templates to create finding-based detections based on your specific requirements.

Detection examples

The following detection SPL templates are included with Splunk Enterprise Security as example queries to help understand the logic behind the detection editor. You can edit these templates to create finding-based detections based on your specific requirements.

  • Finding Group Template - Entity Exceeded MITRE ATT&CK Tactic Threshold

  • Finding Group Template - Entity Exceeded MITRE ATT&CK Technique Threshold

  • Finding Group Template - Entity Exceeded Risk Threshold

  • ESCU - Finding Group - Entity Exceeded Threshold with Multiple Findings

  • Finding Group Template - Entity Exceeded Kill Chain Threshold

Detection templates

Instead of writing inconsistent SPL queries from scratch, you can use tested SPL templates that are provided in the Detection editor to modify or create a finding-based detection. These templates follow the expected pattern for finding-based detections and use the appropriate macros, to which you need to add a basic threshold logic. You can access these templates in the Detection editor in the Search field or the Drill-down searches field in the Analyst queue information section.

When you save the finding-based detection, the [[generate_time_range_for_detection]] gets substituted with [[generate_time_range( "<detection name>" )` | return earliest, latest].

Following is an example of the SPL search template that you can access from the Search field of the Detection editor.

PYTHON 
| tstats `summariesonly` `common_fbd_fields` from datamodel=Risk.All_Risk where
    [[generate_time_range_for_detection]] by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index
| `drop_dm_object_name("All_Risk")`
| `get_mitre_annotations`
| `generate_findings_summary_on_entity`
| where ``` refer to FBD logic documentation for fields to utilize in threshold logic ```

Following is an example of the SPL search template that you can access from the Drill-down searches field in the Analyst queue information of the Detection editor.

PYTHON 
| from datamodel Risk.All_Risk
| where
    [| tstats `summariesonly` `common_fbd_fields` from datamodel=Risk.All_Risk where earliest="$info_min_time$" latest="$info_max_time$" by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index
    | `drop_dm_object_name("All_Risk")`
    | `get_mitre_annotations`
    | `generate_findings_summary_on_entity_for_drilldown_searches`
    | where ``` refer to FBD logic documentation for fields to utilize in threshold logic ``` AND normalized_risk_object="$normalized_risk_object$"
    | eval all_finding_ids=mvappend(intermediate_finding_ids, finding_ids)
    | fields all_finding_ids
    | mvexpand all_finding_ids
    | rename all_finding_ids AS source_event_id]
| `get_correlations`
| `ers_lookup`

Macros to create finding-based detections

Use macros to simplify the process of creating SPL search queries for finding-based detections by standardizing the aggregation of findings and entity-level grouping. Using these macros helps to ensure consistency and maintainability of the SPL query structure​ for finding-based detections.

Macro Description SPL query
generate_findings_summary Differentiates and extracts finding data from notable and risk indexes.​ eval count_findings = if(index == "notable", int_findings_count, 0), finding_ids=if(index == "notable", contributing_event_ids, NIL), count_intermediate_findings = if(index == "risk", int_findings_count, 0), intermediate_finding_ids=if(index == "risk", contributing_event_ids, NIL), intermediate_findings_summary = if(index == "risk", json_object("risk_object", normalized_risk_object, "risk_object_type", risk_object_type, "count", int_findings_count, "risk_score", int_risk_score_sum), NIL), is_finding_group="True"
calculate_findings_fields Aggregates and enriches finding statistics​ definition = stats sum(int_risk_score_sum) as risk_score, dc("annotations.analytic_story") as analytic_story_count, dc("annotations.mitre_attack.mitre_tactic") as mitre_tactic_id_count, dc("annotations.mitre_attack") as mitre_technique_id_count, dc("annotations.kill_chain_phases") as kill_chain_phases_count, dc("annotations.nist") as nist_count, dc("annotations.cis20") as cis20_count, dc(risk_message) as risk_message_count, dc(threat_object_info) as threat_object_count, dc(orig_source) as source_count, values(*) as *, sum(count_findings) as count_findings, sum(count_intermediate_findings) as count_intermediate_findings, sum(eval(count_intermediate_findings+count_findings)) as total_event_count​
generate_findings_summary_on_entity_for_drilldown_searches Creates end-to-end entity-level finding group based on drill-down searches definition = `generate_findings_summary` | `calculate_findings_fields` by `fbd_grouping(normalized_risk_object, risk_object_type)` | eval finding_ids = mvindex(finding_ids, 0, 99) , intermediate_finding_ids = mvindex(intermediate_finding_ids, 0, 99), risk_event_count = count_intermediate_findings | fields - int_risk_score_sum, int_findings_count, contributing_event_ids​

Following is an example of the SPL for a custom finding-based detection that includes thegenerate_time_range macro:

PYTHON 
| tstats `summariesonly` `common_fbd_fields` from datamodel=Risk.All_Risk where [| `generate_time_range("Threat - Finding Group Template - Entity Exceeded Risk Threshold - Rule")` | return earliest, latest ] by All_Risk.normalized_risk_object, All_Risk.risk_object_type, index | `drop_dm_object_name("All_Risk")` | `get_mitre_annotations` | `generate_findings_summary_on_entity` | where risk_score > 100
```

Changes in detection macros in Splunk Enterprise Security version 8.4

In Splunk Enterprise Security version 8.4, the following fields are added to the common_fbd_fields macro in addition to several annotation-related fields:

  • values(All_Risk.threat_object) as threat_object
  • values(All_Risk.src_user) as src_user
  • values(All_Risk.risk_message) as risk_message

In Splunk Enterprise Security version 8.2, these fields were included outside the macro in the finding-based detection searches.

Note: If you created custom finding-based detection searches in Splunk Enterprise Security version 8.2 using these macros, you might see duplication field definition errors.

For example, if you created a custom finding-based detection using the default macros available in Splunk Enterprise Security version 8.2 similar to the following: | tstats `summariesonly` `common_fbd_fields`, values(All_Risk.threat_object) as threat_object from datamodel=Risk.All_Risk...

Then, in Splunk Enterprise Security version 8.4, the threat_object field can be retrieved twice, once inside the macro and once independently on its own, which leads to a duplicate field definition error when invoking the tstats command.

If this occurs, you must delete the values of (All_Risk.threat_object) for the threat_object field from the detection searches since it's now included in the macro.

You don't need to take any action if you used the finding-based detections that were available by default in Splunk Enterprise Security version 8.2 without any modifications. All default detections available in Splunk Enterprise Security are updated. Custom detections don't use the common_fbd_fields macro.

Include the Risk data model in the finding-based detection

Note: New fields are added to the Risk data model to create finding groups. After upgrading to Splunk Enterprise Security version 8.0, the new Risk data model overwrites all customizations that you might have made. Contact Splunk support within 90 days of an upgrade to retrieve any old configurations.

All finding-based detections must have the following base search, which includes the Risk data model:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone from datamodel=Risk.All_Risk

Include specific fields in the search to group findings

Additional fields can be added to the SPL search for the finding-based detection based on specific requirements. Adding fields in the SPL search helps to customize the display of findings in the analyst queue, the Risk Timeline, and the investigations

For example, the following SPL search groups findings by fields such as cim_entity_zone:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10

The following search adds the generate_time_range(<fbd_detection_name> macro and groups findings by cim_entity_zone in the SPL search:

PYTHON 
| tstats `summariesonly` mode(All_Risk.risk_object) as risk_object, sum(All_Risk.calculated_risk_score) as risk_score, count(All_Risk.calculated_risk_score) as risk_event_count, values(All_Risk.annotations.mitre_attack.mitre_tactic_id) as annotations.mitre_attack.mitre_tactic_id, dc(All_Risk.annotations.mitre_attack.mitre_tactic_id) as mitre_tactic_id_count, values(All_Risk.annotations.mitre_attack.mitre_technique_id) as annotations.mitre_attack.mitre_technique_id, dc(All_Risk.annotations.mitre_attack.mitre_technique_id) as mitre_technique_id_count, values(All_Risk.tag) as tag, values(source) as source, dc(source) as source_count, values(All_Risk.risk_object) as all_risk_objects, values(All_Risk.cim_entity_zone) as cim_entity_zone. dc(cim_entity_zone) as cim_entity_zone_count  from datamodel=Risk.All_Risk where [ | generate_time_range(<fbd_detection_name> | return earliest, latest ] where All_Risk.cim_entity_zone=* where cim_entity_zone_count >= 10