Configure index-based search filtering for transparent mode federated providers

Index-based search filtering specifies which indexes are accessible from transparent mode federated providers, which can reduce unnecessary system overhead and improve search performance. Use Splunk platform REST API endpoints to configure this optimization.

Index-based search filtering lets administrators route federated search requests to specific transparent mode federated providers based on the indexes in a search. This gives you more control over which remote Splunk platform deployments receive search requests.

By targeting only the relevant indexes that are accessible from each transparent mode federated provider, index-based search filtering can reduce unnecessary system overhead and improve search performance. It also limits where searches and search logs are sent, which can help minimize exposure of sensitive information and ensure that search logs are available only where they are needed in distributed federated search environments.

Administrators can use the following Splunk platform REST API endpoint parameters to configure index-based provider selection for federated providers operating in transparent mode:

  • The allowIndexBasedProviderFiltering parameter on the data/federated/settings/general endpoint turns on index-based filtering for federated providers.
  • The fedSrchIndexesAllowed parameter on the data/federated/provider/{federated_provider_name} endpoint specifies which indexes cause searches to target that index's federated provider.

How index-based search filtering works

By default, the federated search head dynamically learns which indexes are available on each transparent mode federated provider. It uses this information to route searches to providers that have at least one active index in the search. Providers that don't have any active indexes in the search are excluded.

The fedSrchIndexesAllowed endpoint provides per-provider allowed lists that run on top of the dynamically-learned index-based search filtering. When the allowIndexBasedProviderFiltering parameter is set to true, the federated search head compares the indexes in a search with the fedSrchIndexesAllowed value for each transparent mode federated provider. If at least one active index in the search matches the provider fedSrchIndexesAllowed value, the provider is included in the federated search. If none of the indexes match, the provider is excluded.

For example, if an administrator sets a provider's fedSrchIndexesAllowed parameter to prod_*, the provider is included for the following search:

CODE
index=prod_data OR index=test_data

The same provider is excluded for this search:

CODE
index=test_data OR index=test_user
Note: Index-based search filtering adds to, but does not override, role-based access control (RBAC). A search can target a provider, but users receive results only from indexes that their roles permit them to search.

Set up index-based search filtering

Index-based search filtering specifies which indexes are accessible from transparent mode federated providers, which can reduce unnecessary system overhead and improve search performance. Use Splunk platform REST API endpoints to configure this optimization.

To configure index-based search filtering for transparent mode federated providers, you must first turn on index-based provider filtering using the Splunk platform REST API allowIndexBasedProviderFiltering endpoint parameter. Then, configure the allowed indexes for each transparent mode federated provider using the the Splunk platform REST API fedSrchIndexesAllowed endpoint parameter.

  1. Turn on index-based provider filtering:
    1. Run the following request on the federated search head to set allowIndexBasedProviderFiltering to true on the data/federated/settings/general endpoint:
      CODE
      curl -k -u <username>:<password> -X POST \ https://<host>:<mPort>/services/data/federated/settings/general \ -d allowIndexBasedProviderFiltering=true
  2. Configure allowed indexes for each transparent mode federated provider:
    1. Run the following request on the data/federated/provider/{federated_provider_name} endpoint to set fedSrchIndexesAllowed to the indexes or index patterns that cause searches to target that provider:
      CODE
      curl -k -u <username>:<password> -X POST \ https://<host>:<mPort>/services/data/federated/provider/<federated_provider_name> \ -d 'fedSrchIndexesAllowed=<index_or_pattern>;<index_or_pattern>'

      You can set fedSrchIndexesAllowed to one of the following values:

      • A wildcard ( * ), which matches all federated indexes that the provider is allowed to search.
      • A semicolon-separated list of indexes or index patterns.
      • An empty value, which prevents the provider from being included in the search.
      For example, you can use * as a wildcard in index patterns like this:
      CODE
      -d 'fedSrchIndexesAllowed=prod_*_test;local*'
      This value matches indexes such as prod_security_test, prod_finance_test, and local_telemetry.

The following example turns on index-based provider filtering and configures a transparent mode federated provider named east_provider that is targeted only by searches that include indexes matching prod_* or security_*.

CODE
curl -k -u <username>:<password> -X POST \ https://localhost:8089/services/data/federated/settings/general \ -d allowIndexBasedProviderFiltering=true
CODE
curl -k -u <username>:<password> -X POST \ https://localhost:8089/services/data/federated/provider/east_provider \ -d 'fedSrchIndexesAllowed=prod_*;security_*'