Field filter naming and quantity

Note: Preview features are provided by Splunk to you "as is" without any warranties, maintenance and support, or service level commitments. Splunk makes this preview feature available in its sole discretion and may discontinue it at any time. Use of preview features is subject to the Splunk General Terms.

Before you begin, see Plan for field filters in your organization for important considerations about planning for field filters.

READ THIS FIRST: Should you deploy field filters in your organization?

Field filters are a powerful tool that can help many organizations protect their sensitive fields from prying eyes, but it might not be a good fit for everyone.

If your organization uses downstream configurations, such as accelerated data models, Splunk Enterprise Security (ES) detections using those data models, and user-level search-time field extractions, make sure that you plan around the implications of field filters on those configurations before deploying field filters in your environment. See READ THIS: Downstream impact of field filters.

If your organization runs Splunk Enterprise Security or if your users rely heavily on commands that field filters restricts by default (mpreview and mstats), do not use field filters in production until you have thoroughly planned how you will work around these restricted commands. See READ THIS: Restricted commands do not work in searches on indexes that have field filters.

Naming field filters

Give careful consideration to your naming conventions for your field filters. You can use alphanumeric characters and underscores ( _ ) in names of field filters. Spaces, special symbols, and hyphens ( - ) are not allowed.

Your field filter naming convention is especially important if you want your field filters to be applied in a certain order. Since field filters are applied in lexicographical order, a field filter called 01_filter_ID executes before 02_filter_ID because the 01 prefix is applied before the 02 prefix. Therefore, the final value of the field is determined by 02_filter_ID.

Number of field filters

Once you consider which fields you want to filter with field filters, decide how many field filters you need. If your organization has multiple fields per source type, you might need to configure multiple field filters for each source type.

Your list of field filters should include field filters for indexed fields containing sensitive information you want to protect, as well as field filters for the _raw fields that are used to generate those indexed fields. Typically there will be more field filters for indexed fields, and fewer field filters for _raw fields.

For performance reasons, the total number of field filters in your environment should not exceed 5,000.

Multiple field filters for the same field

If you need to create overlapping _raw field filters to ensure coverage of sensitive data, keep the following in mind.

  • Multiple field filters for the same _raw field are allowed, but only one field filter is applied at a time.

  • Multiple field filters for the same indexed field are allowed provided each field filter has a different target index.

  • In the unlikely case there is a conflict between multiple field filters, the field filter that is last in lexicographical order takes precedence and is applied. For example, if aFieldFilter replaces the value of the field with the string A, and bFieldFilter replaces the value of the field with the string B, and cFieldFilter replaces the value of the field with a hash, then A is replaced with B, and B is hashed. The final field value that appears in the search results is a hash.

  • Field filters for the same indexed field that have different replacements are treated as unique field filters. For example, if you have two field filters for the SSN field, and one field filter is limited by the source type and the other by the host, the field filters overlap, but are distinct.

Next step

Next, plan how you'll define permissions for your field filters. See Roles and permissions.