About Federated Search for Splunk
You can run federated searches to search datasets outside of your local Splunk platform deployment. From your local search head, Federated Search for Splunk gives you a holistic view of datasets across multiple Splunk platform deployments.
Federated search is topology-agnostic, so it works despite the complexity of the Splunk platform deployments involved. You can run a federated search across any remote Splunk Cloud Platform or Splunk Enterprise deployment.
Do you use hybrid search? See Migrate from hybrid search to federated search.
Components of a typical federated search setup
Federated search introduces a set of terms. Familiarize yourself with them before you attempt to dig into setting up and running federated searches.
Federated search
A search of one or more remote datasets on one or more federated providers.
Local deployment
The Splunk platform deployment from which you run federated searches. The federated search head for your federated search resides on your local deployment.
In this context, "local" does not refer to your physical location. If you are in London and are logging into a Splunk platform deployment located in New York City when you run a federated search, that New York City deployment is the local deployment for your federated search.
Federated search head
A search head residing on your local deployment that initiates federated searches.
Federated provider
A local or remote Splunk platform deployment containing the data that you search with your federated searches.
Before you can run federated searches, you must create federated provider definitions on the local deployment. A federated provider definition serves several purposes:
- It enables the federated search head to make network connections to the federated provider and run searches on a remote search head on that provider through a service account.
- It determines whether the federated provider runs in standard or transparent mode, which in turn affects how you write and run federated searches.
For an overview of the standard and transparent modes, see About the standard and transparent modes.
Remote search head
A search head on a federated provider.
Federated index
An index you create on your federated search head to run federated searches over standard mode federated providers. Each federated index maps to a specific remote dataset on a standard mode federated provider. Federated indexes do not ingest data or events. They provide a logical mapping to remote datasets. See Map a federated index to a remote Splunk dataset.
Remote dataset
A dataset on a standard mode federated provider. Currently, only events indexes, metrics indexes, data models, saved searches, and last jobs run by scheduled searches qualify as remote datasets. See Map a federated index to a remote Splunk dataset.
How Federated Search for Splunk works
The process of Federated Search for Splunk works similarly to distributed search. On a distributed search, the initial processing of a search is handled by the indexers of a Splunk platform deployment, and then the results are aggregated on the search head for that deployment to produce a final result set.
However, in Federated Search for Splunk, federated searches are broken up into parts that are processed on a "local" Splunk platform deployment and parts that are processed on one or more remote Splunk platform deployments. Each of these remote Splunk platform deployments is a federated provider.
For example, say you have a simple federated search that involves only one federated provider. In this case, the federated search process sends the remote portion of the search to the federated provider. On the federated provider, the remote search head and its indexers process the search independently, performing a pre-aggregation of the results. The remote search head then sends the results back to the federated search head on the local deployment, where the local search head aggregates the remote results into the final result set for the complete federated search.
The following diagram illustrates a federated search over a remote deployment. The remote deployment is a standard mode federated provider. The federated provider has an events index dataset that is available for federated searches. On the local deployment, a federated index on the federated search head maps to a remote dataset.
A simple federated search for this setup might look like this:
index=federated:provider1_fedindex1 | stats countThis search references a federated index named provider1_fedindex1. The provider1_fedindex1 federated index maps to the remote dataset stored on Federated Provider 1. The remote search head uses this mapping to send back events from its remote index dataset to the federated search head on your local deployment. The federated search head runs the stats count operation on those events. When this stats count aggregation is complete, the federated search head presents the results without additional processing, as there are no additional datasets involved in the search.
See Run federated searches over remote Splunk platform deployments to learn how to write federated searches.
Kinds of federated searches you can set up
This table lists the four kinds of federated searches that you can set up, and the Splunk Enterprise or Splunk Cloud Platform versions that those types of federated searches require.
| Kind of federated search | Local deployment | Remote deployment |
|---|---|---|
| Splunk Enterprise to Splunk Enterprise | Splunk Enterprise (version 8.2.0 or higher) | Splunk Enterprise (version 8.2.0 or higher) |
| Splunk Cloud Platform to Splunk Cloud Platform | Splunk Cloud Platform (version 8.1.2103 or higher) | Splunk Cloud Platform (version 8.1.2103 or higher) |
| Splunk Enterprise to Splunk Cloud Platform | Splunk Enterprise (version 8.2.0 or higher) | Splunk Cloud Platform (version 8.2.2104 or higher) |
| Splunk Cloud Platform to Splunk Enterprise (standard mode federated search only) | Splunk Cloud Platform (version 8.2.2203 or higher) | Splunk Enterprise (version 9.0.0 or higher) |
For more information about federated search modes, see About the standard and transparent modes.
If you have a Splunk Enterprise deployment that is lower than 8.2 and want to run federated searches without upgrading the entire deployment, you can upgrade a single search head in that deployment to 8.2 and run federated searches from that search head.
Splunk Cloud Platform environment and region support
Federated search supports Splunk Cloud Platform deployments in AWS, Google Cloud, and Microsoft Azure.
For the conditions and limitations that apply to region support for federated search in AWS and Google Cloud, including search between regions and the support of regulated cloud environments, see the coverage of federated search in the Splunk Cloud Platform Service Description.
About the standard and transparent modes
When you define a federated provider, you must decide which mode you want that provider to use. Federated provider modes offer different federated search experiences, and you must select the mode that best fits your needs.
There are two federated provider mode options:
- Standard mode
- Choose standard mode if you want to restrict data access to specific remote datasets such as indexes, saved searches, last scheduled search jobs, or data models. Standard mode is the best fit for federated search users who are not migrating from a hybrid search setup.
- Transparent mode
- Choose transparent mode if you want a simple transition to federated search or if you use hybrid searchand want to migrate to federated search. Transparent mode lets you run your hybrid mode searches without syntax changes.
- Transparent mode is available in Splunk Cloud Platform version 8.2.2107 and higher and Splunk Enterprise version 9.0.0 and higher.
Choose the right mode for your environment
How to decide whether to use standard mode or transparent mode across your federated search deployment.
Deciding whether to use standard mode or transparent mode across your federated search deployment depends on how you want your users to interact with remote data and the level of control you require over search execution and resource allocation.
When to choose standard mode
Your organization should choose standard mode if you:
- Require explicit control over when and how remote searches are triggered. By default, standard mode isolates the remote provider and requires explicit permission to access remote provider data sets, such as indexes, saved searches, and data models.
- Want to build new federated search deployments and data access strategies from the ground up without maintaining legacy hybrid search configurations.
- Follow the principle of least privilege and want to ensure that users only interact with remote data sets if they explicitly include the federated: prefix in their searches, which invokes a federated index on the federated search head that maps to a dataset on the remote search head.
In standard mode, you create federated indexes on your local search head that map to specific remote datasets, so searches can run on the remote search head. This approach is ideal for administrators who want to prevent accidental or heavy resource consumption on remote instances, since searches are run only when a user intentionally targets a federated index using the federated: prefix.
Standard mode is also the preferred choice for maintaining a strict separation between local and remote datasets, such as indexes, saved searches, last scheduled search jobs, and data models. It ensures that remote resources are only engaged for specific, targeted searches rather than being part of the general search overhead. This is particularly useful in environments with data sovereignty requirements or where remote providers have limited compute resources that must be protected from high-frequency searches.
Searches that invoke knowledge objects in standard mode can require more knowledge object administration than searches in transparent mode. A standard mode federated search can use knowledge objects from the remote search head, the federated search head, or both, depending on where the knowledge object is used in the search and how the search is optimized. Administrators should review expected standard mode federated searches and ensure that required knowledge objects are available where they are needed. See Manage knowledge objects for standard mode federated providers.
The following are some sample use cases for standard mode federated search:
- Targeted security investigations: A security analyst needs to perform a deep-dive search into a specific remote environment to investigate a rare event without impacting daily dashboard performance.
- Compliance auditing: An auditor needs to run a one-time report against a long-term storage instance or a Splunk environment for a different business unit.
- Cross-deployment correlation: A user needs to correlate data between multiple independent Splunk Cloud Platform stacks or Splunk Enterprise environments for historical or functional reasons.
When to choose transparent mode
Your organization should choose transparent mode if you:
- Want to provide a consistent experience where existing knowledge objects and federated searches remain unchanged. Transparent mode is the best choice for organizations aiming for a unified, seamless user experience across multiple Splunk platform deployments.
- Have specific requirements for knowledge object distribution. In transparent mode, knowledge objects are automatically replicated. The federated search head does not use remote knowledge objects, but instead transmits your local knowledge objects, such as apps and lookups, to the remote deployment. This might be a security concern when connecting deployments across different domains where you might need to restrict access to local assets. For example, the federated search head administrator might not want to expose their local knowledge objects to the remote deployment.
- Prefer users to be able to run existing hybrid searches without needing to make syntax changes to their searches or re-map data sources.
- Plan to migrate to Federated Search for Splunk from a hybrid search. Transparent mode is designed to simplify this transition by mimicking the behavior of your existing hybrid search behavior. Because hybrid search already synchronizes knowledge objects across both on-premises and cloud indexers, transparent mode allows you to migrate to federated search without disrupting your users’ current workflows, searches, or reporting dashboards. In addition, transparent mode creates one search domain with the same knowledge objects, which significantly reduces administrative overhead during the transition.
In this mode, you map remote providers to the local search head so that all searches that are triggered on the local federated search head are also sent to the remote provider. Transparent mode allows you to maintain a "single pane of glass" where the physical location of the data is irrelevant to the end user and remote data feels like it is native to the local environment. Transparent mode also lets users run searches against remote data using standard SPL, such as index=remote_data, without needing to invoke an index using specialized syntax like the federated: prefix that is needed for standard mode federated searches. As long as the remote federated provider service account grants the remote search head access to the indexes specified in the search on the federated search head, and the local user has permissions to its indexes, the remote search head will process those search requests. See Security models for Federated Search for Splunk.
The following are some sample use cases for transparent mode federated search:
- Migration scenarios: Your organization is in the process of moving from a hybrid search setup from an on-premises Splunk Enterprise deployment to Splunk Cloud Platform, and users need to be able to search across both environments during the transition.
- Dashboard portability: You have a large library of pre-built dashboards, saved searches, and alerts that must incorporate data from remote providers without any modification to existing searches.
- Unified app support: Your organization uses premium applications like Splunk Enterprise Security, which often require the transparent mapping of indexes to function correctly across distributed deployments.
Implementation considerations
Keep the following considerations in mind as you plan your rollout of standard or transparent mode in your Federated Search for Splunk deployment.
- Mode selection
- It is not possible to change the mode of the federated provider once it has been created without deleting and recreating it. Therefore, plan your selection carefully during the initial setup phase.
- Security
- If your organization chooses transparent mode federated search, you should do a pre-deployment analysis of the indexes you want to make accessible on remote search heads and make sure that access permissions across your deployment align with your current security policies. See Security models for Federated Search for Splunk.
- Performance
- By default, the local federated search head sends every transparent mode search to all remote search heads, even if every search doesn’t need to be sent to every remote search head, which can cause significant performance and search concurrency consequences on remote search heads. However, you can use granular search targeting controls to alter this behavior and tailor searches to your organization’s specific needs. See Configure role-based access and search targeting for transparent mode federated providers.
Comparison of the standard and transparent modes
Comparison of standard mode and transparent mode in Federated Search for Splunk deployments.
The following table summarizes the differences between the two modes.
| Category | Standard mode federated search | Transparent mode federated search |
|---|---|---|
| Kinds of federated search | Applies to the following kinds of federated search:
|
Applies to the following kinds of federated search:
Note: The Splunk Cloud Platform to Splunk Enterprise kind of federated search does not support transparent mode.
Also applies to Splunk Enterprise to Splunk Cloud Platform federated search, if you are migrating from a hybrid search setup. |
| Provider setup | Requires:
You can associate a single remote deployment with multiple standard mode federated provider definitions. For example, for one remote deployment you might set up different standard mode federated provider definitions for different application contexts. |
Requires federated provider definition only. You can associate a single remote deployment with only one transparent mode federated provider definition. See About creating multiple federated provider definitions for the same host name and port. |
| User permissions applied to remote portion of search | The federated search runs on the federated provider with the permissions of the service account user you define on the federated provider. | The federated search runs on the federated provider with the permissions of the user who initiates the search on the local deployment. |
| Application context of remote portion of search | Uses the application context set in the federated provider definition. | Uses the application context of the local search. |
| Knowledge objects applied to remote portions of searches | Uses knowledge objects that are defined on the remote search head of the federated provider. See Manage knowledge objects for standard mode federated providers. | Through bundle replication, uses knowledge objects from the federated search head of the local deployment. |
| Security | The role-based access control permissions for the service account user on the federated provider determine what your local users can search on the federated provider. In addition, access to federated indexes is role-based, which allows you to restrict your local users' ability to search remote datasets on the federated provider. | The role-based access control (RBAC) permissions for your local users determine what your users can search on the federated provider, with the exception of remote indexes, the access to which is governed by the remote federated provider service account. In addition, to activate transparent mode federated search capabilities for the federated provider, the service account must have the fsh_manage capability. |
| Which local searches run as federated searches on the federated provider? | Only local searches that invoke federated indexes run over remote datasets on federated providers. Searches that do not invoke federated indexes run only on your local deployment. | By default, transparent mode searches can run over the transparent mode federated providers that are selected as default providers for the user’s role. Administrators can use provider-based RBAC and search targeting settings to control which transparent mode providers a role can search and which providers searches target by default. Searches that target unintended providers might reduce performance. See Configure role-based access and search targeting for transparent mode federated providers. |
| Special search processing language (SPL) syntax required? | Yes | No |
| Can send only specific subsearches to the remote search head? | Yes | No |
| Can run entire federated search on the remote search head? | Yes | No |
| Provides separate namespace for remote indexes (to avoid name collisions)? | Yes | No |
| Can run remote saved searches? | Yes | No |
| Can search unaccelerated data models? | Yes. In your search, reference a local federated index that maps to a remote data model on the federated provider. | Yes. In your search, reference a local data model to get data from your local deployment as well as remote data from the federated provider. |
| Can search accelerated data models? | Yes. In your search, reference a local federated index that maps to a remote accelerated data model on the federated provider. | Yes. When you use transparent mode, accelerated data models on your local search head create data model summaries on your local indexers and on the remote indexers of your federated providers. In your search, reference a local accelerated data model to return both local and remote results.
Note: The ability to run transparent mode federated searches over accelerated data models requires that both your local and remote Splunk platform deployments be at either Splunk Cloud Platform 9.0.2303 or higher, or Splunk Enterprise 9.1.0 or higher.
|
| SPL limitations | Standard mode federated search has these SPL limitations:
|
Transparent mode federated search has these SPL limitations:
|
| Dataset availability | You can search the following types of remote datasets on a federated provider:
|
You can search events indexes and metrics indexes on a federated provider. |
About Federated Search for Splunk and Splunk security and IT products
Federated Search for Splunk supports Splunk IT Service Intelligence version 4.16.0 and higher, for transparent mode federated search only. For more information, see New features in Splunk IT Service Intelligence in the ITSI 4.16.0 Release Notes.
Federated Search for Splunk supports Splunk Enterprise Security for transparent mode federated search only, and with recognition of specific limitations. For more information, see the following links:
- If you use Splunk Enterprise Security 7.3.x, see Use federated searches in transparent mode with Splunk Enterprise Security in Use Splunk Enterprise Security.
- If you use Splunk Enterprise Security 8.0.0 or higher, see Use federated searches in transparent mode with Splunk Enterprise Security in Use Splunk Enterprise Security.
Set up Federated Search for Splunk between Splunk platform deployments
Complete the following steps to set up Federated Search for Splunk between a local deployment and a remote deployment.
| Number | Task | For more information |
|---|---|---|
| 1 | Determine the federated provider mode of the remote deployment. | See About the standard and transparent modes |
| 2 | Set up a service account on the remote deployment. | See Service accounts and security for Federated Search for Splunk |
| 3 | Apply a federated provider definition to the remote deployment. Set the provider mode. | See Define a Splunk platform federated provider |
| 4 | If you have defined a standard mode federated provider, define one or more federated indexes for it. | See Map a federated index to a remote Splunk dataset |
| 5 | If you have defined a standard mode federated provider and you intend to run federated searches that are dependent on custom knowledge objects, ensure those knowledge objects exist on the remote search head. | See Manage knowledge objects for standard mode federated providers |
| 6 | Run federated searches. | See Run federated searches over remote Splunk platform deployments |