Create a Databricks connection

Create a Databricks connection in Catalog that authenticates federated searches over Unity Catalog datasets using Delta Sharing.

To set up Federated Search for Databricks on your Splunk Cloud Platform deployment, you must first define an Databricks connection in your Splunk Cloud Platform deployment's Catalog. This connection leverages the Databricks OpenSharing model to authenticate access to Unity Catalog schemas and tables, allowing you to create federated search datasets for those Unity Catalog data objects.

Establishing this connection requires a secure OpenSharing credential file (config.share) that contains a unique bearer token and the sharing endpoint. To obtain this file, you must create a share (a securable object representing the specific Unity Catalog schemas, tables, and views you want to search) and a recipient representing users of your Splunk platform deployment. When you assign the recipient to the share, Databricks generates the credential file within your Databricks workspace.

After you download the credential file from your Databricks workspace, you upload it to your Databricks connection in the Data Management app. This file serves as the primary authentication mechanism, allowing users of your Splunk platform deployment to securely access and search the Unity Catalog data objects included in your Databricks share.

  • Your Splunk Cloud Platform deployment user account must have a role with the edit_connections and edit_datasets capabilities. See Define roles on the Splunk platform with capabilities in Securing Splunk Cloud Platform.
  • Use the Catalog to create and maintain your Databricks connection. See Discover Data Using the Catalog.

  • You must have access to a Microsoft Azure or Amazon Web Services (AWS) Databricks workspace with a runtime of 11.3 LTS or higher that contains the data you want to share and which is assigned to a Unity Catalog metastore. See Enable a workspace for Unity Catalog.
  • Note: The Splunk platform currently only supports federated search of data in Azure and AWS Databricks tables.
    You must have appropriate privileges to access and manage the Databricks data assets that you want your users to be able to search through this connection. You must be an admin for the Unity Catalog metastore where the data you want to share is registered, or you must have the following privileges:
    • CREATE SHARE and CREATE RECIPIENT for the Unity Catalog metastore that contains the data you want to share. See Metastore admins.
    • USE CATALOG and USE SCHEMA for the catalog and schemas that contain the data assets that you want to share. See Manage privileges in Unity Catalog.
  • You must obtain an Databricks OpenSharing credentials file that lets the Splunk platform access your Unity Catalog schemas, tables, and views through the Databricks connection. To obtain this file, complete the following tasks in your Databricks workspace:
    Task Databricks documentation
    Create a share object for OpenSharing. When you create the share, select the data assets you want to add to the share object. Create a share object
    Create a recipient for your share that uses bearer tokens. Ensure the recipient type is Open and the authentication type is Token. Give the token a Token lifetime of up to 365 days. Create a recipient that uses bearer tokens
    Obtain the OpenSharing credentials file. When you create a share recipient, you get an activation link to a page where you can download the credentials file (config.share).
    Note: You can access this activation link until it is used to download the credentials file. After the file is downloaded, Databricks does not display the activation link for the recipient.
    Get an activation link
  1. On your Splunk Cloud Platform deployment, in Splunk Web, select the Catalog icon ( This image shows the Catalog icon, which looks like a document catalog.) to open the Catalog.
  2. On the Connections tab, select Create > Connection to enter the Create connection workflow.
  3. On the Select data store page, select Databricks, and then Next.
  4. On the General page, configure the following settings, and then select Next:
    Setting Description
    Connection name Enter a unique name for the connection. The connection name must start with a lower-case letter and can contain only lower-case alphanumeric characters, underscores, and hyphens.
    Connection description (Optional) Provide a description of the connection.
  5. On the OpenSharing authentication page, upload your OpenSharing credential file.
  6. Take note of the Credential file expiration date that appears after you upload the file.

    When the file expires, this Databricks connection and any datasets associated with it cease to function. You can update the lifetime for the recipient bearer token or rotate bearer tokens to get a new credential file with a new expiration date. See Manage recipient tokens in the Databricks Documentation.

    Note: If you update the token lifetime for a recipient bearer token in Databricks, you do not see the expiration date for the credential file change in your Data Management app UI.
  7. Select Next.
  8. Review your connection details and select Create to create the connection.
You have created a Databricks connection that authorizes access to a Delta Lake share containing one or more Unity Catalog tables or views.
Define a dataset that represents a specific table or view in the Unity Catalog share that your Databricks connection authorizes access to. See Define a Databricks dataset.