Prepare Edge Processor instances for authenticated package downloads

Prepare customer-managed Edge Processor instances in Splunk Enterprise 10.6 before enabling authenticated package downloads.

This procedure applies to Splunk Enterprise deployments that run Edge Processors on customer-managed hosts. It does not apply to Splunk Cloud Platform deployments that download packages from the Splunk Cloud package distribution service.

Make sure that you have the admin role in Splunk Enterprise and administrative access to each instance host.

Why a full reinstallation is required

Each instance includes an Edge Processor component that starts the instance and downloads required software packages. This component cannot update itself, and older versions cannot authenticate package download requests.

Fully uninstalling and reinstalling the instance with the Splunk Enterprise 10.6 installation script replaces the component with an authentication-capable version. Restarting the instance or waiting for an automatic software update does not complete this migration.

Splunk Enterprise 10.6 includes an updated Edge Processor installation script and updated Edge Processor software that support authenticated package downloads. Authentication remains inactive by default in Splunk Enterprise 10.6, so you can prepare your deployment before a future release enables it by default.

CAUTION: Update every affected Edge Processor instance before you enable package download authentication or upgrade to a release that requires it. Otherwise, an instance might fail the next time it automatically restarts or downloads an update, interrupting data ingestion and potentially causing data loss.

Identify affected instances

After you upgrade Splunk Enterprise to version 10.6, review the Edge Processors page in the Data Management app. If one or more instances require reinstallation, the page displays migration guidance and identifies the affected Edge Processors and instances.

The following indicators mean that action is required:

  • An Edge Processor with at least one affected instance is labeled Needs attention.
  • An affected instance is labeled Needs reinstallation.

If you do not see these indicators, your instances already support authenticated package downloads and do not need to be reinstalled for this migration.

Choose a preparation option

Choose one of the following options:

  1. Reinstall affected instances and use authenticated package downloads is the recommended preparation path. Fully uninstall and reinstall each instance labeled Needs reinstallation using the latest Splunk Enterprise 10.6 installation script. After you verify all affected instances, you can optionally enable package download authentication before a future release enables it by default.

  2. Keep package download authentication inactive. Create an app-local restmap.conf override that explicitly keeps authentication inactive when a future release enables it by default. This option leaves Edge Processor packages available without authentication from the Splunk management port. Confirm that this configuration aligns with your organization's security requirements.

Reinstall affected instances and enable package download authentication

Fully uninstall and reinstall each instance labeled Needs reinstallation using the latest Splunk Enterprise 10.6 installation script. After you update all affected instances, enable authentication and validate the secured download flow.

Plan a rolling reinstallation

An Edge Processor is a logical group that can have one or more instances. Each standalone instance runs on a separate host. During a rolling reinstallation, a multi-instance Edge Processor can contain both updated and affected instances.

Plan the reinstallation so that the remaining instances can process incoming data. Before taking an instance offline:

  • Confirm that the remaining instances are Healthy and can process incoming data.
  • Confirm how upstream senders or load balancers can stop routing data to an offline instance and fail over to the remaining instances.
  • Determine the minimum number of instances required to handle peak traffic. This number is your capacity floor.
  • Confirm that you have administrator access to Splunk Enterprise and each instance host.

Calculate the maximum batch size as follows:

CODE 
maximum instances offline = total instances - capacity floor

For example, if an Edge Processor has 10 instances and requires 8 instances to handle peak traffic, reinstall no more than 2 instances at a time.

For a single-instance Edge Processor, add an updated second instance to the same Edge Processor and confirm that it is Healthy and receiving traffic before you remove the original instance. If you cannot add temporary capacity, schedule an ingestion interruption and account for the buffering and delivery behavior of each upstream sender.

Review the failover and buffering behavior of each upstream sender. Splunk forwarders must have another available target and sufficient queue capacity. HEC and syslog senders require a resilient client configuration or load balancer. UDP does not guarantee delivery, so provide another collection path or minimize the interruption.

  1. Select your desired batch. On the Edge Processors page, identify instances labeled Needs reinstallation. Select a batch that does not exceed the maximum number of instances that you can take offline.
    For a single-instance deployment, first add an updated instance to the same Edge Processor or schedule an ingestion interruption, as described in the Plan a rolling reinstallation section above.
  2. CAUTION: Taking an instance offline while it is still receiving data can cause data loss. Data held in memory or in persistent queues might be discarded.
    Remove the instances from active traffic. Stop upstream senders or reconfigure them to send data to other instances. Verify that no data is being sent to the instances that you plan to reinstall.
    Verify that the selected instances are no longer receiving new data and that traffic has moved to the remaining instances before you uninstall them.
  3. Fully uninstall each selected instance using the supported uninstallation procedure.
    1. On the Edge Processors page, in the row that lists the Edge Processor, select the Actions icon and then select Install/uninstall.
    2. Expand Step 1: Run commands to install/uninstall instances.
    3. Select Uninstall, and then select Copy to clipboard.
    4. On the instance host, open a command-line interface and run the copied command.
    If systemd manages the splunk-edge service, follow Manage and uninstall Edge Processors for the supported commands and verification steps.
  4. Confirm that uninstallation is complete before reinstalling.
    • On the Edge Processors page, select View instances from the Edge Processor's action menu and verify that the instance no longer appears in the table.
    • Verify that the splunk-edge process or service is no longer running on the host.
    • Verify that no sender or load balancer is routing data to the instance.

    Do not delete the logical Edge Processor. The reinstalled instance must join the same Edge Processor so that it receives the same pipelines and shared configuration.

  5. Install the updated instance into the same logical Edge Processor.
    1. On the Edge Processors page, in the row that lists the same Edge Processor, select the Actions icon and then select Install/uninstall.
    2. Expand Step 1: Run commands to install/uninstall instances.
    3. Select Install, and then select Copy to clipboard.
    4. On the instance host, open a command-line interface and run the copied Splunk Enterprise 10.6 installation commands.
    Always generate a new installation command. Do not reuse a command saved before the upgrade or from a previous installation attempt.
  6. Confirm that the instance returned to service.
    • Confirm that the instance appears in the instances table.
    • Confirm that its status is Healthy.
    • Confirm that it is not labeled Needs reinstallation.
    • Confirm that load balancer health checks have returned it to active rotation, when applicable.
    • Confirm that data is flowing through the instance's pipelines as expected.

    Observe the instance for a stabilization period appropriate for your environment before starting the next batch.

  7. Repeat the reinstallation process for all affected instances until no instance in the Splunk Enterprise deployment is labeled Needs reinstallation.

    Do not enable package download authentication until every affected instance has been reinstalled and verified.

    CAUTION: Enabling authentication while an affected instance remains can prevent that instance from downloading or restarting required software. Data ingestion through the instance can stop and might result in data loss.
  8. After you reinstall and verify all affected instances, you can optionally enable package download authentication before it is enabled by default in a future release. To enable authentication early, choose one of the following methods.

    Choose one method and complete its instructions.

    Enable authentication from the first-time setup page

    1. In Splunk Web, open the Splunk Pipeline Builders app.

    2. Navigate directly to the app's setup page by appending /setup to the app URL. For example:

      CODE 
      https://<splunk-web-host>:<port>/<locale>/app/splunk_pipeline_builders/setup
    3. Under Authentication for Edge Processor package downloads, turn on Authentication required.
    4. Select Save. Saving writes the app-local requireAuthentication = true setting.
    5. Restart Splunk Enterprise to apply the restmap.conf change.
    6. Return to the setup page and confirm that authentication is enabled.
    7. Confirm that every Edge Processor instance remains Healthy and that data continues to flow as expected.

    Enable authentication in restmap.conf

    1. Create or update the following file:

      CODE 
      $SPLUNK_HOME/etc/apps/splunk_pipeline_builders/local/restmap.conf

    2. Add the following stanza and setting:

      CODE 
      [script:edge-binary-server]
requireAuthentication = true
    3. Restart Splunk Enterprise, then confirm that all instances remain Healthy and that data continues to flow.

Keep package download authentication inactive

Create an app-local restmap.conf override that explicitly keeps authentication inactive when a future release enables it by default. This option leaves Edge Processor packages available without authentication from the Splunk management port. Confirm that this configuration aligns with your organization's security requirements.

Keep package download authentication inactive if you choose that preparation option.
  1. Create or update the following app-local file:
    CODE 
    $SPLUNK_HOME/etc/apps/splunk_pipeline_builders/local/restmap.conf
  2. Add the following stanza and setting:
    CODE 
    [script:edge-binary-server]
requireAuthentication = false
  3. Restart Splunk Enterprise after creating or changing the override.

The app-local value takes precedence over the value shipped with the app. It persists through upgrades until an administrator changes or removes it.

The Edge Processors page continues to display migration guidance and actions because the affected instances have not been reinstalled. After you create the override and restart Splunk Enterprise, you can ignore this migration guidance.

Important: While this override is present, Edge Processor packages remain downloadable without authentication from the Splunk management port. Confirm that this configuration aligns with your organization's security requirements.

Verify the preparation

Perform the following steps to verify and troubleshoot your instance preparation.

Confirm that preparation is complete for either path:

  • No Edge Processor or instance is labeled Needs attention or Needs reinstallation.
  • Every Edge Processor instance is Healthy, and data continues to flow through all associated pipelines.
  • If you enabled authentication, the first-time setup page shows Authentication required as enabled.
  • If you kept authentication inactive, requireAuthentication = false exists in the app-local restmap.conf file, and Splunk Enterprise has been restarted since the configuration was added or changed.

Troubleshoot the migration

If a reinstalled instance does not become Healthy, complete these steps:

  1. Confirm that the host can connect to the Splunk management port and that intervening proxies or firewalls allow the connection.
  2. On the Edge Processors page, select Install/uninstall from the Edge Processor's action menu and generate new installation commands. Run the new commands on the instance host.
  3. Review the instance logs for Edge Processor component, package download, or authentication errors.
  4. Confirm that the instance was installed into the intended logical Edge Processor.
  5. Contact Splunk Support if the problem continues.

If you enabled authentication before updating every instance, set requireAuthentication to false in the app-local restmap.conf file and restart Splunk Enterprise to restore package downloads while you troubleshoot. This restores unauthenticated package downloads but does not update the affected instances. Complete the reinstallation before enabling authentication again.