Search-time order and DMA

Field filters process field values before downstream search-time operations that depend on those field values. This ordering can affect DMA searches and the summaries they generate.

If a data model acceleration search uses a sensitive field in commands such as eval, where, search, lookup, or calculated fields, a field filter can change the value before those commands run. Any downstream logic that depends on the original value can produce search results that you don't expect.

For example, if a data model acceleration search evaluates a derived field based on whether a sensitive field contains a specific value, replacing that sensitive value can change the derived field. If an ES detection or tstats search later depends on the derived field, the detection result can change.

The same principle applies after tstats returns filtered values from a DMA summary. Commands that run after tstats operate on the replacement value for non-exempt users. Test searches that use filtered fields in downstream comparisons, calculations, grouping, alert conditions, and dashboards.

Here's another important example to consider. Say you have a data model called DM_search_identitytest that uses the following summarization search to create summaries to detect malicious users:

CODE 
| summarize tstats=t override=partial manual_rebuilds=f max_time=3600 poll_buckets_until_maxtime=f id=DM_search_identitytest 
    [ search (index=* OR index=_*) (index=identitytest) 
    | eval nodename="identitytest" 
    | eval malicious_user=if(searchmatch("val=86"),"true","false") 
    | rename Code AS identitytest.Code malicious_user AS identitytest.malicious_user 
    | fields nodename, _time, host, source, sourcetype, identitytest.Code, identitytest.malicious_user]

When this summarization search runs without a field filter, the malicious_user field evaluates to true, alerting you to a possible security breach.

Then, you create a field filter on the sensitive field called val. When the DM_search_identitytest data model's search runs with the field filter on the val field, the result of the search command is the redacted value, which is a value other than 86. When the next command in the pipeline, the eval command, evaluates the val field to the redacted field value, the malicious_user field is now false. Now when you have alerts with tstats searches on the accelerated data model set on the malicious_user field that trigger on true instead of false, you might not be alerted and, therefore, miss potential security breaches.