Verify your Ingest Processor and pipeline configurations
The Ingest Processor starts processing and routing your data after you've completed the following steps:
- Set up the Ingest Processor service.
- Add a destination to route your data. You can skip this step if you want to route data to the Splunk Cloud Platform deployment connected to the Ingest Processor tenant.
- Create and apply at least one pipeline.
- Configure at least one data source to send data to the Splunk Cloud Platform deployment that's connected to the Ingest Processor tenant. See the Splunk Cloud Platform Get Data In manual for more information.
After you complete these steps, the Ingest Processor processes data and sends it to a destination based on the data processing instructions defined in the applied pipelines.
To confirm that data is actually flowing through the Ingest Processor, you can view the inbound and outbound data metrics of the Ingest Processor. As an additional confirmation step, you can verify your data at its destination. For example, you can search an index to confirm that your data is reaching that index as expected. See the sections that follow for more detailed guidance on verifying that the Ingest Processor is working as expected.
View the inbound and outbound data metrics of the Ingest Processor
In the Ingest Processor service, you can open a detailed view of the Ingest Processor that displays information such as the amount of data that it is receiving and sending out to destinations.
To access this detailed view, navigate to the Ingest Processor page. For more information, see View data flow information about the Ingest Processor.
If the data flow metrics do not match what you expect, then verify your configurations.
Search for your data in the destination index
Use Splunk Cloud Platform to search for the data that you sent through your Ingest Processor.
- Log in to the Splunk platform deployment that you configured your Ingest Processor to send data to.
- From the Apps panel in Splunk Web, select Search & Reporting.
- Search the destination index to confirm that it contains the expected events. For example, if you configured your Ingest Processor pipeline to send data to an index named
my_index, then use the following search criteria to find your data:CODEindex="my_index"
If your processed data is not showing up at its destination as expected, then verify your configurations.
Confirming and troubleshooting your configurations
If you encounter unexpected results or behavior while using the Ingest Processor solution, make sure that your data source, source type, pipeline, and destination are configured correctly. Specifically, verify the following:
- Your data source is configured to send data to the Splunk Cloud Platform deployment that's connected to the tenant, and the Splunk Cloud Platform deployment is configured to receive that data.
-
The source type of the data that you want to process is listed on the Source types page in Splunk Cloud Platform, and this source type is configured with the appropriate event-breaking definitions. For more information, see Create source types in the Splunk Cloud Platform Get Data In manual.
-
Your pipeline is configured correctly. Make sure that your pipeline isn't filtering out data that you want to keep.
Note: When your pipeline is opened for editing, you can generate a preview for each destination to confirm how your pipeline processes data. See Getting sample data for previewing data transformations and Create pipelines for Ingest Processors for more information. - The destination used by your pipeline is configured with the correct connection settings and credentials.
If the problems persist, do the following:
- Review the logs for your Ingest Processor pipeline and the associated supervisor to identify the cause of the problem. See View logs for the Ingest Processor solution.
- Review the troubleshooting documentation for potential solutions or workarounds. See Troubleshoot the Ingest Processor solution.