Send data from Ingest Processor to Microsoft Azure

Send data from Ingest Processor to Azure Blob Storage or Azure Data Lake Storage containers by using Microsoft Azure datasets as pipeline destinations.

Note: In the Controlled Availability release stage, Splunk products may have limitations on customer access, features, maturity, and regional availability. For additional information on Controlled Availability please contact your Splunk representative.

Sending data to Microsoft Azure is supported in Ingest Processors that are associated with Splunk Cloud Platform version 10.4.2604 and higher.

To send data from Ingest Processor to an Azure Blob Storage container or an Azure Data Lake Storage container, you must do the following:

  1. In the Data Management app on Splunk Cloud Platform, create a connection that provides access to Azure storage containers by authenticating through a Microsoft Entra app registration. For more information, see Create a Microsoft Azure connection for Ingest Processor pipelines.

  2. In the Data Management app on Splunk Cloud Platform, create a dataset that represents the location in Azure where you want to send data. For more information, see Create a Microsoft Azure dataset for Ingest Processor pipelines.

  3. In the Ingest Processor service, create a pipeline that uses the Microsoft Azure dataset as a destination. For more information, see Create pipelines for Ingest Processor.

    Note: To ensure that the events you send to the dataset are compatible with federated searches, there are several best practices that you need to follow when configuring your pipeline. For more information, see Best practices for sending data from Ingest Processor to a dataset.

  4. In the Ingest Processor service, apply the pipeline to the Ingest Processor. For more information, see Apply a pipeline.

When you apply that pipeline to Ingest Processor, it starts sending the data that it receives to your Azure container. In Azure, this data is identified by a file path and name that is constructed using auto-generated values from the system as well as some of the values that you specify in the connection and dataset configuration.

Note: You are responsible for the egress charges incurred when sending data from a pipeline to Microsoft Azure.