How Federated Search for Splunk works

Components of a typical federated search setup

Federated search introduces a set of terms. Familiarize yourself with them before you attempt to dig into setting up and running federated searches.

Term Description
Federated search

A search of one or more remote datasets on one or more federated providers.
Local deployment

The Splunk platform deployment from which you run federated searches. The federated search head for your federated search resides on your local deployment.

In this context, "local" does not refer to your physical location. If you are in London and are logging into a Splunk platform deployment located in New York City when you run a federated search, that New York City deployment is the local deployment for your federated search.
Federated search head

A search head residing on your local deployment that initiates federated searches.
Federated provider

A local or remote Splunk platform deployment containing the data that you search with your federated searches.

Warning: Federated Search for Splunk is designed to work only with remote deployments running the Splunk platform. Using Federated Search for Splunk to connect to any other third-party systems, including providers, is not supported.

Before you can run federated searches, you must create federated provider definitions on the local deployment. A federated provider definition serves several purposes:

  • It enables the federated search head to make network connections to the federated provider and run searches on a remote search head on that provider through a service account.
  • It determines whether the federated provider runs in standard or transparent mode, which in turn affects how you write and run federated searches.

For an overview of the standard and transparent modes, see About the standard and transparent modes.

See Define a Splunk platform federated provider.
Remote search head

A search head on a federated provider.
Federated index

An index you create on your federated search head to run federated searches over standard mode federated providers. Each federated index maps to a specific remote dataset on a standard mode federated provider. Federated indexes do not ingest data or events. They provide a logical mapping to remote datasets. See Map a federated index to a remote Splunk dataset.
Remote dataset

A dataset on a standard mode federated provider. Currently, only events indexes, metrics indexes, data models, saved searches, and last jobs run by scheduled searches qualify as remote datasets. See Map a federated index to a remote Splunk dataset.

How Federated Search for Splunk works

The process of Federated Search for Splunk works similarly to distributed search. On a distributed search, the initial processing of a search is handled by the indexers of a Splunk platform deployment, and then the results are aggregated on the search head for that deployment to produce a final result set.

However, in Federated Search for Splunk, federated searches are broken up into parts that are processed on a "local" Splunk platform deployment and parts that are processed on one or more remote Splunk platform deployments. Each of these remote Splunk platform deployments is a federated provider.

For example, say you have a simple federated search that involves only one federated provider. In this case, the federated search process sends the remote portion of the search to the federated provider. On the federated provider, the remote search head and its indexers process the search independently, performing a pre-aggregation of the results. The remote search head then sends the results back to the federated search head on the local deployment, where the local search head aggregates the remote results into the final result set for the complete federated search.

The following diagram illustrates a federated search over a remote deployment. The remote deployment is a standard mode federated provider. The federated provider has an events index dataset that is available for federated searches. On the local deployment, a federated index on the federated search head maps to a remote dataset.

Note: The federated index is included in the diagram because the federated provider in this example is a standard mode federated provider. Transparent mode federated providers do not require federated indexes.

This diagram illustrates a federated search over a remote deployment. The remote deployment is a standard mode federated provider. The federated provider has an events index dataset that is available for federated searches. On the local deployment, a federated index on the federated search head maps to a remote dataset.

A simple federated search for this setup might look like this:

CODE 
index=federated:provider1_fedindex1 | stats count

This search references a federated index named provider1_fedindex1. The provider1_fedindex1 federated index maps to the remote dataset stored on Federated Provider 1. The remote search head uses this mapping to send back events from its remote index dataset to the federated search head on your local deployment. The federated search head runs the stats count operation on those events. When this stats count aggregation is complete, the federated search head presents the results without additional processing, as there are no additional datasets involved in the search.

See Run federated searches over remote Splunk platform deployments to learn how to write federated searches.