The guidelines shown in the Sizing guidelines based on daily data ingestion volume section on this page were determined based on internal performance testing using the following configurations:

• The Edge Processor was installed on an Amazon EC2 C5 instance. For information about the specifications, see “Amazon EC2 C5 Instances” on the Amazon Web Services website.

• The Splunk-to-Splunk (S2S) protocol was used for data transmission.

• The data was sent from a forwarder to the Edge Processor, and then from the Edge Processor to a Splunk platform deployment.

• The incoming data consisted of Palo Alto Network logs with an average event size of 400-600 bytes. These logs were consistent with the pan:firewall source type and looked like the following:

  unused,2025/06/10 18:23:49,001234567890,TRAFFIC,URL,1.0,2025/06/10 18:23:49,112.52.238.98,68.25.111.114,,,block-social-media-apps-strict,carol.networking@example.org,,,dns-long-request-lookup,vsys1,DMZ-Frontline-Application-Network,Untrust-Network-Segment,ethernet1/1,ethernet1/2,,12345,0,54823,443,12345,54321,0xABCDEF,tcp,deny,www.example-longdomain.org,,,informational,1001,,US,Global,,,,,,,,,"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)",text/html,,,,,,,Company1,Company2,Company3,Company4,vsys1,firewall-1,,,,,,,proxy-tag-info,,high,
  
  unused,2025/06/10 18:23:49,001234567890,TRAFFIC,URL,1.0,2025/06/10 18:23:49,108.181.41.167,82.57.184.190,,,allow-internal-web-access-to-dmz,bob.superadmin@example.com,,,ssl-encrypted-browsing,vsys1,Internal-Zone-Long-Label,Untrust-Network-Segment,ethernet1/1,ethernet1/2,,12345,0,54823,443,12345,54321,0xABCDEF,tcp,drop,www.example-longdomain.org,,,informational,1001,,US,Global,,,,,,,,,"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)",text/html,,,,,,,Company1,Company2,Company3,Company4,vsys1,firewall-1,,,,,,,proxy-tag-info,,high,
  
  unused,2025/06/10 18:23:49,001234567890,TRAFFIC,URL,1.0,2025/06/10 18:23:49,55.212.51.43,38.66.82.16,,,restrict-vpn-access-on-weekends,bob.superadmin@example.com,,,http-cleartext,vsys1,Trust-Extended-Zone-Name,External-SaaS-Zone,ethernet1/1,ethernet1/2,,12345,0,54823,443,12345,54321,0xABCDEF,tcp,allow,www.example-longdomain.org,,,informational,1001,,US,Global,,,,,,,,,"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)",text/html,,,,,,,Company1,Company2,Company3,Company4,vsys1,firewall-1,,,,,,,proxy-tag-info,,high,
  

• The data was processed by a pipeline that extracted fields, filtered data, and normalized data. This pipeline configuration represents mid-to-high complexity data transformations. For example:

  function remove_readable_timestamp($source) {
  return | from $source
  | eval readable_time_regex = "\\w{3}\\s\\d{2}\\s\\d+:\\d+:\\d+"
  | eval _raw=replace(_raw, readable_time_regex, "")
  | fields -readable_time_regex
  }
  
  function route_test($source) {
  return | from $source
  | eval route_win=1
  }
  
  function external_ip($source) {
  return | from $source
  | eval cond_ip=if(cidrmatch("10.0.0.0/8",dest_ip) OR cidrmatch("172.16.0.0/12",dest_ip) OR cidrmatch("192.168.0.0/16",dest_ip), "internal", "external")
  | eval cond_vw=if(match(dest_zone, "String$"), 1, 0)
  | where NOT(cond_ip="internal" AND cond_vw="1")
  }
  
  $pipeline = | from $source 
  | rex field=_raw /^^.*?,.*?,.*?,.*?,.*?,.*?,.*?,.*?,(?P<dest_ip>.*?),.*?,.*?,(?P<rule>.*?),.*?,.*?,.*?,.*?,(?P<src_zone>.*?),(?P<dest_zone>.*?),/
  | where not(dest_ip="192.168.0.1" )
  | external_ip
  | eval index = "main"
  | eval ep="1"
  | eval src_user = if(src_user = "", "unknown", src_user)
  |eval _time=strptime(generated_time + " PST", "%Y/%m/%d %H:%M:%S")
  // Replace the Raw with a friendly string, to prevent the "Palo Alto Networks" TA from re-writing the time incorrectly
  | eval _raw = src_user + " on " + src_ip +" via policy " + rule + " accessing " + misc
  |fields -future_use3, future_use1, receive_time
  | rex field=_raw /^^(?P<future_use1>.*?),(?P<receive_time>.*?),(?P<serial_number>.*?),(?P<type>.*?),(?P<log_subtype>.*?),(?P<version>.*?),(?P<generated_time>.*?),(?P<src_ip>.*?),(?P<dest_ip>.*?),(?P<src_translated_ip>.*?),(?P<dest_translated_ip>.*?),(?P<rule>.*?),(?P<src_user>.*?),(?P<dest_user>.*?),(?P<app>.*?),(?P<vsys>.*?),(?P<src_zone>.*?),(?P<dest_zone>.*?),(?P<src_interface>.*?),(?P<dest_interface>.*?),(?P<log_forwarding_profile>.*?),(?P<future_use3>.*?),(?P<session_id>.*?),(?P<repeat_count>.*?),
  ,(?P<src_port>.*?),(?P<dest_port>.*?),(?P<src_translated_port>.*?),(?P<dest_translated_port>.*?),(?P<session_flags>.*?),(?P<transport>.*?),(?P<action>.*?),(?P<misc>.*?),(?P<threat>.*?),(?P<raw_category>.*?),(?P<severity>.*?),(?P<direction>.*?),(?P<sequence_number>.*?),(?P<action_flags>.*?),(?P<src_location>.*?),(?P<dest_location>.*?),(?P<future_use4>.*?),(?P<content_type>.*?),(?P<pcap_id>.*?),(?P<file_hash>.*?),(?P<cloud_address>.*?),(?P<url_index>.*?),(?P<user_agent>.*?),(?P<file_type>.*?),(?P<xff>.*?),(?P<referrer>.*?),(?P<sender>.*?),(?P<subject>.*?),(?P<recipient>.*?),(?P<report_id>.*?),(?P<devicegroup_level1>.*?),(?P<devicegroup_level2>.*?),(?P<devicegroup_level3>.*?),(?P<devicegroup_level4>.*?),(?P<vsys_name>.*?),(?P<dvc_name>.*?),(?P<future_use5>.*?),(?P<src_vm>.*?),(?P<dest_vm>.*?),(?P<http_method>.*?),(?P<tunnel_id>.*?),(?P<tunnel_monitor_tag>.*?),(?P<tunnel_session_id>.*?),(?P<tunnel_start_time>.*?),(?P<tunnel_type>.*?),(?P<threat_category>.*?),(?P<content_version>.*?),(?P<future_use6>.*?)/
  | into $destination;
  

When using the configurations described above to test the performance of a single-instance Edge Processor on host machines with different specifications, the following results were observed:

When testing the performance of a multi-instance Edge Processor, the following results were observed:

In all of these cases, the Edge Processor was able to ingest and process the data without holding it in the persistent queue or back pressuring it.

This table describes the following system requirements for processing a given amount of incoming data per day:

• The number of Edge Processors that must be installed
• The number of instances that each Edge Processor must be scaled out to
• The amount of CPU and memory that the host machine of each Edge Processor must have

• Installation requirements for Edge Processors
• Set up an Edge Processor
• Add more instances to an Edge Processor