About the Edge Processor Solution

The Edge Processor solution is a data processing solution that works at the edge of your network. Use the Edge Processor solution to filter, mask, and transform your data close to its source before routing the processed data to external environments.

The Edge Processor solution is suitable for Splunk Enterprise administrators who use forwarders, syslog devices, or HTTP Event Collector (HEC) to get data into their deployments. You can use the Edge Processor solution on a Splunk Enterprise instance running version 10.0.0. Edge Processor is compatible with supported forwarder-indexer combinations where forwarders run on version 8.2.x or higher. See Compatibility between forwarders and Splunk Enterprise indexers for more information on forwarder-indexer version compatibility.

By paring down and sanitizing data before sending it out to Splunk indexes or Amazon S3 buckets, you can reduce data storage costs and help prevent confidential data from leaving your network. With the Edge Processor solution, you can also manage your data processing configurations and monitor your data ingest traffic through a centralized Splunk service.

Note: For information about the latest product updates, see Release Notes.

Compare Ingest Actions to the Edge Processor solution

Ingest Actions is another Splunk data transformation service. Ingest Actions and the Edge Processor solution can largely handle the same use cases. For example, both allow you to filter verbose data sources, such as Windows event logs, to retain selected events or content within an event. Both the Edge Processor solution and Ingest Actions let you match a certain event code, mask the extensive message field at the end of Windows events, and route an unfiltered copy of data to an AWS S3 bucket.

The Edge Processor solution offers a centralized control plane to manipulate your data pipelines through Search Processing Language, version 2 (SPL2) while Ingest Actions offers a graphical user interface over existing props and transforms so that you can create rulesets to affect the data transformation. The following table provides a side-by-side comparison of the two services:

Edge Processor solution Ingest Actions
Platform availability Is available in Splunk Cloud Platform and on Splunk Enterprise. Is natively available in both Splunk Enterprise and Splunk Cloud Platform.
Cost All current Edge Processor features are free to all Splunk Cloud and Splunk Enterprise users. All current Ingest Actions features are free to all Splunk Enterprise and Splunk Cloud users.
Method of access Requires activation for Splunk Cloud Platform users. Ask a Splunk sales representative for access to the Edge Processor solution if you are already a Splunk Cloud Platform user. See Enable Edge Processor on Splunk Enterprise to get started with Edge Processor on Splunk Enterprise. Is natively available in both Splunk Enterprise and Splunk Cloud Platform.
Transformation capabilities Relies on Splunk Search Processing Language, version 2 (SPL2), which allows you to create tightly defined logic to transform data through pipelines. Transforms data through rulesets, which are defined through drop-down menu options, offering more ease of use but less detailed options.
Closeness to the data source Is usually closer to the data source when you transform your data. It represents another forwarding tier. Is farther away from the data source if you configure it directly on the indexing tier. If you configure Ingest Actions on the heavyweight forwarding tier, it is equally close to the data source as the Edge Processor solution.
User interface Has a graphical user interface (UI) and allows you to compare your inbound and outbound data. For example, you can preview what percentage of your inbound data becomes your outbound data based on how you code your pipeline logic. You can also see all your Edge Processors in one place and deploy your pipeline logic to your different Edge Processors in one place. Has a graphic user interface (UI) and includes data previews before implementing your code. You can visualize directly how events are edited before and after you deploy your ruleset. However, your Ingest Action rulesets might not be visible all in one place. Your Ingest Actions ruleset are available on the indexing or heavyweight forwarding tier that you implemented them on.
Sources Can receive data from these sources:
  • Splunk universal forwarders
  • Heavyweight forwarders
  • HTTP clients and logging applications through HEC
  • Syslog
  • Splunk Connect for Syslog (SC4S)
Can receive data from any source supported by the Splunk platform. You cannot deploy Ingest Actions on a universal forwarder, but you can receive data from a universal forwarder. You can deploy Ingest Actions on a heavyweight forwarder.
Destinations The Edge Processor solution can route to the following destinations:
  • Amazon S3
  • Splunk Enterprise
  • Splunk Cloud Platform
Ingest Actions can route to the following destinations:
  • Amazon S3
  • Splunk Enterprise
  • Splunk Cloud Platform

  • Local file system

How to use the Edge Processor solution

The Edge Processor solution combines Splunk-managed services, on-premises data processing software, and Search Processing Language, version 2 (SPL2) pipelines to support data processing at the edge of your network. The following table describes how these components work together and how you can use them:

Component Description Usage
Edge Processor A data processing engine that allocates resources for processing and routing data You install Edge Processors on machines in your local network. Edge Processors provide an on-premises data plane that lets you reduce and sanitize your data before sending it outside of your network.
Edge Processor service A service that enables managing Edge Processors Splunk hosts the Edge Processor service as part of Splunk Enterprise. The Edge Processor service provides a control plane that lets you deploy configurations, monitor the status of your Edge Processors, and gain visibility into the amount of data that is moving through your network.
Pipeline A set of data processing instructions written in SPL2, which is the data search and preparation language used by Splunk software In the Edge Processor service, you create pipelines to specify what data to process, how to process it, and what destination to send the processed data to. Then, you apply pipelines to your Edge Processors to configure them to start processing data according to those instructions.

By using the Edge Processor solution, you can process data in your own local network while also managing and monitoring your data ingest ecosystem from a self-managed service.

This diagram provides an overview of the following:

This diagram shows how the Edge Processor service on Splunk Enterprise works with the Edge Processors installed on the customer's network. Data is generated by a source, collected by agents such as forwarders, sent to an Edge Processor for processing, and then routed to a destination..

Start using the Edge Processor solution

To start using the Edge Processor solution, you need to set up a data management control plane within your Splunk Enterprise deployment. See Set up a data management control plane for more information.

If you are the first Edge Processor user on your data management control plane, you need to complete a one-time setup procedure to fully activate the Edge Processor service. See First-time setup instructions for the Edge Processor solution for more information.

To start processing data at the edge of your network, you first need to install an Edge Processor on a machine in your network. Then, specify how you want to process and route your data by creating pipelines using SPL2. Finally, configure your data sources to send data to your Edge Processor. For more guidance on getting started, see Quick start: Process and route data using Edge Processors.

For in-depth information about the Edge Processor solution, see the How the Edge Processor solution works chapter.

See also

See the following documentation for more information about the Edge Processor solution and other Splunk software that works in conjunction with the Edge Processor solution.

For this information Refer to this documentation
System requirements that apply to the data management control plane Setup prerequisites
Complete information about the supported SPL2 commands and functions The following pages in the SPL2 Search Reference:
How to configure Splunk forwarders The Forwarding Data manual
How to configure HEC Set up and use HTTP Event Collector in Splunk Web