About Federated Search for Databricks

Run federated searches from your Splunk platform deployment over Databricks tables stored remotely in Unity Catalog using SPL2 search commands and syntax.

Federated Search for Databricks lets you run federated searches from your Splunk platform deployment over Microsoft Azure and Amazon Web Services (AWS) Databricks tables stored remotely in Unity Catalog. When you run these federated searches, you'll use familiar SPL2 search commands and syntax.

What you need to get started

To get started with federated search of Databricks data, you must have the following things:
  • You must have a Splunk Cloud Platform (SCP) deployment.

  • Use the Catalog to create and maintain your Databricks federated dataset. See Discover Data Using the Catalog.

  • Your user account on the SCP deployment must have a role with the edit_connections and edit_datasets capabilities. See Define roles on the Splunk platform with capabilities in the Splunk Cloud Platform Manage Users and Security manual.

  • You must have access to an Azure or AWS Databricks workspace with a runtime of 11.3 LTS or higher that contains the data you want to share and which is assigned to a Unity Catalog metastore. See Enable a workspace for Unity Catalog.

    Note: The Splunk platform currently only supports federated search of data in Azure and AWS Databricks tables.

Checklist of tasks to set up Federated Search for Databricks

Use this checklist to guide you through the cross-account setup of Federated Search for Databricks.

Step Task Description
1 Create a Databricks connection You upload an OpenSharing credentials file to a connection to give it the ability to authenticate federated searches over Databricks datasets from your Splunk platform deployment.
2 Define a Databricks dataset When combined with a connection, a Databricks federated dataset provides the ability to run searches over a specific Unity Catalog table in your Databricks workspace.
3 Give your users role-based access control of federated datasets After you have successfully created a Databricks federated dataset, give your users role-based access to it.
4 Write and run federated searches over federated datasets with SPL2 Run federated searches over your new Databricks federated dataset with SPL2.