Use cases for field filters with DMA

There are two main use cases for protecting sensitive fields in searches on accelerated data models depending on whether or not a DMA summary has already been built.

There are two main use cases for protecting sensitive fields in searches on accelerated data models, depending on when field filters are created and applied to the accelerated data model:

Use case 1: Field filters are created and applied before a summary is built or rebuilt.

If the field filter is active before the summary is built or rebuilt, the summary will be populated with replacement values from the field filter instead of original sensitive values.

Use case 2: Field filters are created and applied after a summary has already been built with original sensitive values.

If the summary already contains original sensitive values, field filters can protect the values returned to non-exempt users when tstats reads from the summary.

Use case 1: Field filters are in use before the DMA summary is built or rebuilt

This is the simplest and most straightforward use case. Use this approach when you need the accelerated data model summary itself to contain replacement values instead of original sensitive values. In this use case, field filters are applied during summary generation before field values are written to the DMA summary. As a result, the DMA summary stores the replacement value, not the original sensitive value. After the summary is built, neither exempt nor non-exempt users can retrieve the original value from that DMA summary, because the original value was never written to the DMA summary.

In this case, you create the field filter before data model acceleration runs or before you rebuild the summary. Configure the field filter on the source field that the data model acceleration search sees, such as ssn, and make sure the roles used by the internal summarization search are not exempt from that field filter.

When the acceleration search runs with the field filter applied, the summary is populated with the replacement value. Later, when users run tstats searches against the summary, the tstats search returns the replacement value because the original value was not written into that DMA summary.

Use this approach when you need stronger protection for data stored in DMA summary files. If an existing summary already contains original values, rebuild the summary or wait for Splunk software to rebuild or age out the old summary data according to your configured schedule.

Role exemptions can bypass search-time filtering, but they can't recover original values from a DMA summary that never stored those original values. An exempt user might still see original values when searching raw events or when a tstats search includes summariesonly=f and falls back to raw data for buckets that are not covered by the summary. Test searches that read from the DMA summary and searches that fall back to raw data for exempt and non-exempt roles.

The following diagram describes use case 1. Since the field filter was in place before acceleration, the summary was built with redacted values.

Use case 1 showing a field filter applied before data model acceleration. The ssn_redact filter redacts the raw ssn field from the identity_events index before the identity DMA summary is built, so the DMA bucket stores only identity.ssn with a redacted value. When non-exempt and exempt users run tstats values(identity.ssn), both read the accelerated summary and receive only the redacted value; the original SSN is unavailable to all roles.

Use case 2: Field filters are in use after a summary has already been built

Note: Use this approach when you need search-result protection and controlled role-based exceptions. Do not use this approach as your only control if your security requirement is to remove original values from the summary files on disk.

Use this approach to protect sensitive values that are already in a DMA summary, when an accelerated data model summary already contains original sensitive values and you need to prevent non-exempt users from seeing those values in tstats results. When a non-exempt user runs a tstats search against the accelerated data model summary, the field filter replaces the original value with the configured replacement value in the search result. The original sensitive field value can still exist in the DMA summary .tsidx files on disk.

This use case supports role-based exceptions. Roles that are exempt from the field filter can still access the sensitive fields that the field filter protects since the DMA summary contains the original field values. If a user has a role that is listed under Roles without filter for the field filter, the filter is not applied for that user, and the user can see the original value from the summary. Only roles that are allowed to see the original sensitive value should be exempt from the field filter.

If you use this approach, it's important to create the field filter on the field name that the tstats command reads from the DMA summary. For example, if tstats returns the sensitive field as user_activity.ssn, configure the field filter for user_activity.ssn.

The following diagram describes use case 2. The DMA summary already contains original values when the field filter is created. The field filter protects tstats results at search time, so non-exempt users see redacted values while exempt users can still see original values. Original values remain in the summary .tsidx files until the DMA summary is rebuilt; field filters applied at search-time do not remove them from the summary.

Use case 2 showing a field filter applied after data model acceleration. The identity DMA summary is built from the identity_events index before the ssn_redact field filter is applied, so the DMA bucket stores identity.ssn with the original value. When users run tstats values(identity.ssn), the field filter is applied during the search on identity.ssn. The user with a non-exempt role can see the redacted identity.ssn value, while a user with an exempt role can see the original value.