Limitations on using field filters in your environment
Before you begin, see Plan for field filters in your organization for important considerations about planning for field filters.
READ THIS FIRST: Should you deploy field filters in your organization?
Field filters are a powerful tool that can help many organizations protect their sensitive fields from prying eyes, but it might not be a good fit for everyone.
If your organization uses downstream configurations, such as accelerated data models, Splunk Enterprise Security (ES) detections using those data models, and user-level search-time field extractions, make sure that you plan around the implications of field filters on those configurations before deploying field filters in your environment. See READ THIS: Downstream impact of field filters.
If your organization runs Splunk Enterprise Security or if your users rely heavily on commands that field filters restricts by default (mpreview and mstats), do not use field filters in production until you have thoroughly planned how you will work around these restricted commands. See READ THIS: Restricted commands do not work in searches on indexes that have field filters.
Field filter limitations
Consider the following issues when setting up your environment to take advantage of field filters.
For additional limitations about searching with field filters, see Use field filters in searches.
Federated search
Field filters can't be used to protect sensitive fields in remote datasets using federated search.
Reports and scheduled searches
Field filters redact sensitive fields dynamically when a search is run based on user roles. However, protection from field filters only applies while the search is running; once a search completes and its results are stored, field filters don’t protect the results if they can be accessed by other users or roles. This means that you can’t share a report that automatically redacts sensitive fields for some roles but gives full access to the data to other roles. As a result, you should carefully plan your report sharing and permissions strategy to ensure sensitive data is properly protected and not exposed through saved search artifacts.
In addition, if the owner of a scheduled search has broad access, such as an admin role that is exempt from a field filter, users with read access to the saved search results might see sensitive data in plain text unless permissions are carefully restricted. This is because scheduled searches run with the permissions of the report owner, so saved results inherit owner permissions.
To address these limitations, administrators must manage read permissions on scheduled searches to avoid unintended data exposure, and create separate reports for different role requirements to ensure there are no security gaps or data leaks. For example, if you want different users to have different access to sensitive data protected by field filters, you should create two separate reports:
- A protected report: a filtered report generated from a search where the field filter is in place for all users and sensitive data is redacted.
- In this case, the search runs under a role that is not exempt from the field filter, so all sensitive fields are redacted when the search runs. Because the search results don’t contain any sensitive data, it is safe to share the report more broadly. You should set permissions on this report so users who can’t access sensitive information have read access to the report.
- An unprotected report: an unfiltered report that displays sensitive data for certain highly privileged roles that are exempt from the field filter.
- In this case, an unfiltered report is generated from a search executed under a highly privileged role that is exempt from the field filter. When a role that is exempt from the field filter runs the search, sensitive fields are not redacted, so the search results and the report generated from those results contain all sensitive values. Since the stored report now includes unprotected data, this version of the report must be shared only with users or roles that have the privilege to view sensitive data. You should set permissions on this report so those users who should have access to the sensitive information have read access to the report.
Summary indexes
Summary indexes generated by scheduled reports can expose sensitive index information that you might not want unprivileged roles to be able to access. If you collect sensitive summary index information that you want to filter with field filters, carefully consider which highly trusted roles can create those summary indexes.
By default, all reports run with the permissions of the report "owner", which is the person who created the report. To ensure that unprivileged users don't have access to sensitive data, set up your summary indexes using a role restricted by field filters. For example, if your field filter applies to the support role, you must also use the support role when you set up your summary indexes.
See Determine whether to run reports as the report owner or report user in the Reporting Manual and Create a summary index in Splunk Web in the Knowledge Manager Manual.
Example
If the role you use to set up your scheduled searches is exempt from a field filter, but your users hold a different role that is restricted by the field filter, you might expose your confidential data to people who aren't allowed to access that information.
Consider what might happen if you set up a scheduled search using the admin role, which you have exempted from a field filter that replaces the value of the host name field with a SHA-256 hash. When a user holding a role called test that is not exempt from the field filter runs a search on the index, this is what they might see:
| Time | Event |
|---|---|
| 2/28/23
10:15:00.000 AM |
02/28/2023 10:15:00 -0800, search_name=TestSummary, search_now=1646075700.000, info_min_time=1646072100.000, info_max_time=1646075700.000, info_search_time=1646075700.607, orig_host="test-mbp-571a7", action=upload_mmdb_files, psrsvd_v=1, psrsvd_gc=9, testField="testValue"
host = c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d |
Notice that the host name shows up in two places in the summary index. The value for the first instance of the host field is test-mbp-571a7, and the value for the second instance is a hash. In this case, using one role, the admin role, to set up the scheduled search and then running the search using a different role, the test role, that is restricted by field filters, resulted in exposing the data that was supposed to be obfuscated with the hash value.
If the same test role that is configured with field filters is also used to set up the scheduled search, this is what the user who holds the test role sees when they run the search:
| Time | Event |
|---|---|
| 2/28/23
10:15:00.000 AM |
02/28/2023 10:15:00 -0800, search_name=TestSummary, search_now=1646075700.000, info_min_time=1646072100.000, info_max_time=1646075700.000, info_search_time=1646075700.607, orig_host="c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d", action=upload_mmdb_files, psrsvd_v=1, psrsvd_gc=9, testField="testValue"
host = c1fe560f903b48c96812691329a783a9333414bbf7ac26c26bf41cebbaf79b6d |
Now both instances of the host field in the summary index are obfuscated with the hash value. As a result, sensitive data about the true value of the host name is no longer exposed to unprivileged users.