Set up an Edge Processor in a Docker container

After provisioning a shared service principal, perform the following tasks to set up a containerized Edge Processor instance.

1. Compile a list of ingress ports your instance uses to receive ingested data. To do so, navigate to the Edge Processors page in your tenant's web UI, and select Shared Settings in the top right corner.

2. Use Docker's -p flag to define your ports for all supported ingestion types. Map each port to the corresponding port on the host machine.

3. Next, find the GROUP_ID value, which associates new instances with a specific Edge Processor. Select a processor listed on the Edge Processors page of your tenant's web UI, then copy its ID field, such as 431e1ead-fd5b-4af8-ac89-ccae2ae81eda. This value also appears at the end of the page's URL.

4. Run the following command to mount the principal.yaml file to the /opt/splunk-edge/etc/principal.yaml path in the container, publishing the ports gathered previously, and running the image:

   CODECopy

   docker run -d \ -e TENANT=<tenant-id> \ -e GROUP_ID=<edge-processor-id> \ -e TOKEN=<access-token> \ -e MACHINE_HOSTNAME=$(hostname) \ -v $(pwd)/principal.yaml:/opt/splunk-edge/etc/principal.yaml \ -p <port-1>:<port-1> ··· -p <port-n>:<port-n> \ splunk/edge-processor:<image-tag>

   docker run -d \
     -e TENANT=<tenant-id> \
     -e GROUP_ID=<edge-processor-id> \
     -e TOKEN=<access-token> \
     -e MACHINE_HOSTNAME=$(hostname) \
     -v $(pwd)/principal.yaml:/opt/splunk-edge/etc/principal.yaml \
     -p <port-1>:<port-1> ··· -p <port-n>:<port-n> \
     splunk/edge-processor:<image-tag>

Because the generated service principal is designed to be shared by multiple Edge Processor instances, you can start more containers by rerunning the previous docker run command:

docker run -d \
  -e TENANT=<tenant-id> \
  -e GROUP_ID=<edge-processor-id> \
  -e TOKEN=<access-token> \
  -e MACHINE_HOSTNAME=$(hostname) \
  -v $(pwd)/principal.yaml:/opt/splunk-edge/etc/principal.yaml \
  -p <port-1>:<port-1> ··· -p <port-n>:<port-n> \
  splunk/edge-processor:<image-tag>

<start new section here - see drafts later on the page>

Modifying GROUP_ID between runs registers instances with different Edge Processors in your tenant under the same service principal.

As a best practice, restrict service principals to 1 service principal per processor to avoid unnecessary confusion and potential rate limiting. You can use a single service principal across different host machines if the same principal.yaml file is mounted in each container, but this is not a best practice.

Avoiding port conflicts on a single host

Using multiple instances on a single host affects port availability. When running multiple containers on the same host machine, port conflicts become a bottleneck since each published container port must bind to a unique port on the host.

To prevent conflicts, publish the container port without pinning it to a specific host port by using -p <container_port> instead of -p <host_port>:<container_port>. When you omit the host port, Docker selects an available, ephemeral port at run time and publishes the container port accordingly.

This approach can hinder discoverability. Because host ports are assigned dynamically, clients don't know where to route traffic ahead of time. As a result, you must query the running containers to compile a per-host inventory of published ports, then distribute traffic among them using a load balancer or client-side routing.

docker inspect $(docker ps -q) -f '
  {{- $id := slice .Id 0 12 -}}
  {{- range $cport, $bindings := .NetworkSettings.Ports -}}
    {{- if $bindings -}}
      {{- range $b := $bindings -}}
  {{ printf "%s: %s -> %s:%s\n" $id $cport $b.HostIp $b.HostPort }}
      {{- end -}}
    {{- end -}}
  {{- end -}}
'

Using statically assigned host ports

docker run -p 9001:8088 {···}
docker run -p 9002:8088 {···}
docker run -p 9003:8088 {···}

Using a single service principal per processor

Modifying GROUP_ID between runs registers instances with different Edge Processors in your tenant under the same service principal.

As a best practice, restrict service principals to 1 service principal per processor to avoid unnecessary confusion and potential rate limiting. You can use a single service principal across different host machines if the same principal.yaml file is mounted in each container, but this is not a best practice.