Using source types to break and merge data in Edge Processors

The source type is one of the default fields that Splunk software assigns to events. It identifies the kind of data that you are working with and indicates the original source of the data.

You can use the source type configurations from Splunk Enterprise to specify how the Edge Processor breaks and merges the inbound stream of data into distinct events. The event breaking and merging operations defined in your source type configurations are applied to inbound data if it meets the following:

  • The sourcetype value of an event matches the name of a source type configuration in the Edge Processor service.
  • The inbound data isn't already event-broken through other means, such as by the EVENT_BREAKER configuration in a universal forwarder.

Sync source types to Edge Processor

Sync the source type configurations from your Splunk Enterprise deployment to bring them over into the Edge Processor service and make them available for use by Edge Processor instances.

Prerequisites

Users must have source types created or set up on the Splunk Enterprise instance that hosts their data management control plane to use in Edge Processor before syncing. To configure source types to be used by Edge Processors, do the following:

  1. In the Splunk Enterprise instance that hosts your data management control plane, navigate to the Create Source Type window.
  2. In the Advanced section, specify the following line breaking options to ensure your source types successfully sync to the Edge Processor service:
    • LINE_BREAKER
    • SHOULD_LINE_MERGE
    • BREAK_ONLY_BEFORE
    • MAX_EVENTS
    • TRUNCATE

    If these configurations are not set, the source type will not be able to sync. These configurations can be set with either the default or non-default values.

If a source type in Splunk Enterprise specifies non-default values for any line breaking options other than LINE_BREAKER, SHOULD_LINE_MERGE, BREAK_ONLY_BEFORE, MAX_EVENTS, or TRUNCATE, then the source type is not synced over.

All other line breaking options are unsupported, and must be unspecified or set to their default values. The default values will not affect the line breaking behavior in the Edge Processor. For more information about these line breaking options and their default values, see props.conf in the Splunk Enterprise Admin Manual.

For more information on creating new source types in Splunk Enterprise, see Manage source types in the Splunk Enterprise Getting Data In manual.

Steps

Once the source type is configured in Splunk Enterprise to sync to your Edge Processor, sync the source type configurations with the Edge Processor service by doing the following:

  1. Navigate to the the Edge Processor service on your data management control plane in Splunk Enterprise.
  2. In Edge Processor, navigate to the Synced source types page.
  3. Select the Sync source types button.
  4. If the sync fails with the error message "Unable to synchronize sourcetypes," select Retry.
  5. Once successfully synced, you will receive a message indicating how many of your source types were synced and all synced source types will be listed on the Synced source types page.

    Note: Up to 6000 source types can be synced to your Edge Processor service.