Sizing guidelines for Edge Processors

Scale your Edge Processors to provide the necessary support for your incoming data.

The system requirements for your deployment of the Edge Processor solution can vary depending on the volume of data being processed and the complexity of the processing operations involved. In order to support higher volumes of incoming data, or more complex data transformations such as field extraction and normalization, you might need to install your Edge Processors on host machines with higher specifications or scale out your Edge Processors by adding more instances.

To determine how to scale your Edge Processor deployment, start by estimating the amount of incoming data that your Edge Processors will ingest and process per day. Then, refer to the table in the Sizing guidelines based on daily data ingestion volume section on this page.

Note: Be aware that there is a soft limit on the maximum number of Edge Processor instances that can be supported. See Tested and recommended service limits (soft limits) in the Splunk Cloud Platform Service Details for more information.

The sizing guidelines were determined based on internal performance testing. For information about the configurations and specifications used in the performance tests, see Context for sizing guidelines.

For information on how to apply the sizing guidelines to Edge Processors on a Kubernetes cluster, see Mapping the sizing guidelines to a Kubernetes cluster.

Context for sizing guidelines

The guidelines shown in the previous section were determined based on internal performance testing using the following configurations:

  • The Edge Processor instances were installed on Amazon EC2 C5a and C6a instances. For information about the specifications, see the following pages on the Amazon Web Services website:

  • Each Edge Processor was processing data from 50 applied pipelines, and each pipeline was partitioned on a single source.

  • The Splunk-to-Splunk (S2S) protocol was used for data transmission. The data was sent from a heavy forwarder to the Edge Processor, and then from the Edge Processor to a Splunk platform deployment through an S2S connection.

    • The test setup consisted of 80 to 500 heavy forwarder instances.

    • The heavy forwarders handled the line-breaking of the incoming events, so the data processing workload of the Edge Processors did not include line-breaking.

  • The incoming data consisted of Palo Alto Network logs and Windows Event logs with an average event size ranging from 70 bytes to 8.6 kilobytes.

  • The data was processed by pipelines that enriched the events with additional information by parsing and extracting fields from the events.

For more information about the data and pipeline configuration used in the performance tests, see the expandable sections that follow. Each section shows a representative example of an event and the configuration of the pipeline that was used during performance testing.

Pipeline 1: Extract customer and device metadata from Palo Alto Networks syslog messages, classify the messages by log type, and enrich the logs with host name values before routing them to the destination.

Sample event:

JSON 
{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:27.087874108Z"}},"message":"<14>1 2026-04-01T15:24:27+00:00 hostname - - - - 1,2026/04/01 15:24:26,010108001430,TRAFFIC,end,2561,2026/04/01 15:24:26,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXX,,,ssl,vsys1,internal,external,ethernet1/1.1,ethernet1/2.2,log-forwarding-main,2026/04/01 15:24:26,71538790,1,65406,23775,443,0x40047a,tcp,allow,24074,8570,15504,44,2026/04/01 14:54:17,1808,computer-and-internet-info,,72638213781273912,0x8000000000000000,1.1.1.1,United States,,22,22,aged-out,5267,53,52,0,vsys1,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,sdnwjq-sdsa-4533-ab21-adnandjk88,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:27.085+00:00,,,encrypted-tunnel,networking,browser-based,4,\"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use\",,ssl,no,no,0"} 

{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:28.511259452Z"}},"message":"<14>1 2026-04-01T15:24:28+00:00 hostname - - - - 1,2026/04/01 15:24:08,010701010641,TRAFFIC,end,2817,2026/04/01 15:24:08,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXXAware - Managed DNS,,,dns-base,vsys1,l3-internal,l3-external,ae3.15,ethernet2/28.10,log-forwarding-main-dns-only,2026/04/01 15:24:27,2793178772,1,37709,53,7102,53,0x400019,udp,allow,293,113,180,2,2026/04/01 15:23:37,0,any,,sdvdsvd,0x8000000000000000,1.1.1.1,United States,,1,1,aged-out,5267,53,50,0,vsys1,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,c6db958a-f3e9-4264-9e87-39f37973446b,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:28.200+00:00,,,infrastructure,networking,network-protocol,3,\"used-by-malware,has-known-vulnerability,pervasive-use\",dns,dns-base,no,no,0,NonProxyTraffic,"} 

{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:28.510794234Z"}},"message":"<14>1 2026-04-01T15:24:28+00:00 hostname - - - - 1,2026/04/01 15:24:08,010701010641,TRAFFIC,end,2817,2026/04/01 15:24:08,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXX,,,ssl,vsys3,internal,external,ethernet1/25.88,ethernet1/26.96,log-forwarding-main,2026/04/01 15:24:27,2958683727,1,58877,443,35059,443,0x40047a,tcp,allow,9346,3759,5587,27,2026/04/01 15:23:48,16,urlcat_RITM2954011,,7603808268918733999,0x8000000000000000,1.1.1.1,XXX,,14,13,tcp-fin,5267,4553,4556,0,cit,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,a132b316-bdc6-403a-9153-e1575e552284,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:28.200+00:00,,,encrypted-tunnel,networking,browser-based,4,\"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use\",,ssl,no,no,0,NonProxyTraffic,"}

Pipeline configuration:

PYTHON 
$pipeline =  
| from $source  
| eval customer=json_extract(_raw,"metadata.customer") 
| eval aideid = json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval sourcetype = case (like(_raw, "%TRAFFIC%"),"pan:traffic", like(_raw, "%THREAT%"),"pan:threat", like(_raw, "%SYSTEM%"),"pan:system" ,like(_raw, "%CONFIG%"), "pan:config",like(_raw, "%CORRELATION%"), "pan:correlation",like(_raw, "%HIPMATCH%"), "pan:hipmatch",like(_raw, "%GLOBALPROTECT%"), "pan:globalprotect",like(_raw, "%USERID%"), "pan:userid",true,"pan:log") 
| rex field=_raw /^<\d+>\d+\s+\S+\s+(?P<host>\S+)/ 
| into $destination;

Pipeline 2: Extract Windows Event logs from JSON, parse the XML-formatted event messages to identify the channel, and enrich the logs with host name values and a channel-based source identifier before routing them to the destination.

Sample event:

JSON 
{"agent":{"name":"server1"},"process":{"name":"-","pid":0,"executable":"-"},"winlog":{"computer_name":"comp1","process":{"pid":1128,"thread":{"id":15412}},"keywords":["Audit Success"],"logon":{"id":"1X11X111X","type":"Network"},"channel":"Security","event_data":{"LogonGuid":"{00000000-0000-0000-0000-000000000000}","TargetOutboundDomainName":"-","VirtualAccount":"%%1843","LogonType":"3","TransmittedServices":"-","SubjectLogonId":"0x0","TargetOutboundUserName":"-","KeyLength":"128","LmPackageName":"NTLM V2","TargetLogonId":"1X11X111X","RestrictedAdminMode":"-","TargetLinkedLogonId":"0x0","SubjectUserName":"-","ElevatedToken":"%%1843","SubjectDomainName":"-","TargetUserName":"abc123","ImpersonationLevel":"%%1833","TargetDomainName":"MS","LogonProcessName":"NtLmSsp ","SubjectUserSid":"S-1-0-0","TargetUserSid":"X-1-1-11-111111111-1111111111-1111111111-1111111","AuthenticationPackageName":"NTLM"},"opcode":"Info","version":2,"record_id":8525424013,"task":"Logon","event_id":"4624","provider_guid":"{54849625-5478-4994-a5ba-3e3b0328c30d}","activity_id":"{11111111-XX11-1111-1111-11111XX1XX11}","api":"wineventlog","provider_name":"Microsoft-Windows-Security-Auditing"},"log":{"level":"information"},"@metadata":{"beat":"winlogbeat","type":"_doc","version":"8.8.2"},"event_original":"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>4624</EventID><Version>2</Version><Level>0</Level><Task>12544</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2026-03-05T20:55:44.999795600Z'/><EventRecordID>8525424013</EventRecordID><Correlation ActivityID='{11111111-XX11-1111-1111-11111XX1XX11}'/><Execution ProcessID='1128' ThreadID='15412'/><Channel>Security</Channel><Computer>comp1</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>S-1-0-0</Data><Data Name='SubjectUserName'>-</Data><Data Name='SubjectDomainName'>-</Data><Data Name='SubjectLogonId'>0x0</Data><Data Name='TargetUserSid'>X-1-1-11-111111111-1111111111-1111111111-1111111</Data><Data Name='TargetUserName'>abc123</Data><Data Name='TargetDomainName'>MS</Data><Data Name='TargetLogonId'>1X11X111X</Data><Data Name='LogonType'>3</Data><Data Name='LogonProcessName'>NtLmSsp </Data><Data Name='AuthenticationPackageName'>NTLM</Data><Data Name='WorkstationName'>WORKSTATION1</Data><Data Name='LogonGuid'>{00000000-0000-0000-0000-000000000000}</Data><Data Name='TransmittedServices'>-</Data><Data Name='LmPackageName'>NTLM V2</Data><Data Name='KeyLength'>128</Data><Data Name='ProcessId'>0x0</Data><Data Name='ProcessName'>-</Data><Data Name='IpAddress'>1.1.1.1</Data><Data Name='IpPort'>50044</Data><Data Name='ImpersonationLevel'>%%1833</Data><Data Name='RestrictedAdminMode'>-</Data><Data Name='TargetOutboundUserName'>-</Data><Data Name='TargetOutboundDomainName'>-</Data><Data Name='VirtualAccount'>%%1843</Data><Data Name='TargetLinkedLogonId'>0x0</Data><Data Name='ElevatedToken'>%%1843</Data></EventData><RenderingInfo Culture='en-US'><Message>An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\t-\r\n\tAccount Domain:\t\t-\r\n\tLogon ID:\t\t0x0\r\n\r\nLogon Information:\r\n\tLogon Type:\t\t3\r\n\tRestricted Admin Mode:\t-\r\n\tVirtual Account:\t\tNo\r\n\tElevated Token:\t\tNo\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tX-1-1-11-111111111-1111111111-1111111111-1111111\r\n\tAccount Name:\t\tabc123\r\n\tAccount Domain:\t\tMS\r\n\tLogon ID:\t\t1X11X111X\r\n\tLinked Logon ID:\t\t0x0\r\n\tNetwork Account Name:\t-\r\n\tNetwork Account Domain:\t-\r\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x0\r\n\tProcess Name:\t\t-\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\tWORKSTATION1\r\n\tSource Network Address:\t1.1.1.1\r\n\tSource Port:\t\t50044\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tNtLmSsp \r\n\tAuthentication Package:\tNTLM\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\tNTLM V2\r\n\tKey Length:\t\t128\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.</Message><Level>Information</Level><Task>Logon</Task><Opcode>Info</Opcode><Channel>Security</Channel><Provider>Microsoft Windows security auditing.</Provider><Keywords><Keyword>Audit Success</Keyword></Keywords></RenderingInfo></Event>","source":{"port":50044,"ip":"1.1.1.1","domain":"WORKSTATION1"},"message":"An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tNo\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tX-1-1-11-111111111-1111111111-1111111111-1111111\n\tAccount Name:\t\tabc123\n\tAccount Domain:\t\tMS\n\tLogon ID:\t\t1X11X111X\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tWORKSTATION1\n\tSource Network Address:\t1.1.1.1\n\tSource Port:\t\t50044\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.","@timestamp":"2026-03-05T20:55:44.999Z","related":{"ip":"1.1.1.1","user":"abc123"},"event":{"code":"4624","provider":"Microsoft-Windows-Security-Auditing","created":"2026-03-05T20:55:45.707Z","kind":"event","module":"security","action":"logged-in","type":["start"],"category":["authentication"],"outcome":"success"},"fields":{"log_topic":"microsoft-windows"},"user":{"domain":"MS","name":"abc123","id":"X-1-1-11-111111111-1111111111-1111111111-1111111"}}

Pipeline configuration:

PYTHON 
$pipeline = | from $source  
| eval host=json_extract(_raw,"winlog.computer_name") 
| eval _raw=json_extract(_raw,"event_original") 
| rex field=_raw /<Channel>(?P<win_channel>[^<]+)<\/Channel>/ 
| eval source="XmlWinEventLog:"+win_channel 
| into $destination;

Pipeline 3: Extract customer and device metadata from JSON, isolate the message content, parse an ISO 8601 timestamp from it, and route the enriched records to the destination.

Sample event:

JSON 
{"metadata":{"proxy":{"address":"2.2.2.2","port":"500"},"aideid":"APP1","source":{"address":"src1","port":"00000","type":"syslog"},"timestamp":{"producer_process":"2026-02-17T18:32:46.110886807Z"},"customer":"Customer1"},"message":"<13>1 2026-02-17T12:09:37.000-06:00 comp1 InTrust - 4769 [InTrust.Predefined@3973 RecordKey=\"1427165792\" EventType=\"8\" UserName=\"\" UserDomain=\"\" SourceName=\"Microsoft-Windows-Security-Auditing\" ComputerName=\"comp1\" NumericCategory=\"14337\" Category=\"Kerberos Service Ticket Operations\" EventID=\"4769\" GatheringEventLog=\"Security\" GatheringComputer=\"XXXXXXXXXXXXXXX\" GatheringDomain=\"MS\" PlatformID=\"500\" VersionMajor=\"10\" VersionMinor=\"0\" ComputerType=\"93489320\" DatasourceType=\"{X9X5X7X2-5X01-41X7-9X36-X562XXXXXA9}\" Environment=\"9X442XXX-XXX2-4X79-9013-053XX225XXX0\"][InTrust.Named@3973 Account_Name=\"XXX.XXX.COM\" Account_Domain=\"XXX.XXX.COM\" Service_Name=\"XXXXXXXXXXXXX\" Service_ID=\"MS\\\\DXBFSDJKFBESJK\" Source_Network_Address=\"::ffff:3.3.3.3\" Source_Port=\"63888\" Ticket_Options=\"0x40800000\" Failure_Code=\"0x0\" Ticket_Encryption_Type=\"0x12\" Transited_Services=\"-\" Failure_Reason=\"Successful\" Who=\"XXX.XXX.COM\" WhoDomain=\"XXX.XXX.COM\" What=\"Kerberos Service Ticket Operations\" Where=\"comp1\" Severity=\"High\" Where_From=\"::ffff:3.3.3.3\" Result=\"Successful\"] A Kerberos service ticket was requested.\r\n\r\nAccount Information:\r\n\tAccount Name:\t\tXXX.XXX.COM\r\n\tAccount Domain:\t\tXXX.XXX.COM\r\n\tLogon GUID:\t\t{8X932478-X75X-8XXX-3X5X-796X7276117X}\r\n\tMSDS-SupportedEncryptionTypes:\tN/A\r\n\tAvailable Keys:\tN/A\r\n\r\nService Information:\r\n\tService Name:\t\tDXBFSDJKFBESJK\r\n\tService ID:\t\tMS\\DXBFSDJKFBESJK\r\n\tMSDS-SupportedEncryptionTypes:\t0x1F (DES, RC4, AES128-SHA96, AES256-SHA96)\r\n\tAvailable Keys:\tAES-SHA1, RC4\r\n\r\nDomain Controller Information:\r\n\tMSDS-SupportedEncryptionTypes:\t0x1F (DES, RC4, AES128-SHA96, AES256-SHA96)\r\n\tAvailable Keys:\tAES-SHA1, RC4\r\n\r\nNetwork Information:\r\n\tClient Address:\t\t::ffff:3.3.3.3\r\n\tClient Port:\t\t63888\r\n\tAdvertized Etypes:\t\n\t\tAES256-CTS-HMAC-SHA1-96\n\t\tAES128-CTS-HMAC-SHA1-96\n\t\tRC4-HMAC-NT\r\n\r\nAdditional Information:\r\n\tTicket Options:\t\t0x40800000\r\n\tTicket Encryption Type:\t0x12\r\n\tSession Encryption Type:\t0x12\r\n\tFailure Code:\t\t0x0\r\n\tTransited Services:\t-\r\n\r\nTicket information\r\n\tRequest ticket hash:\t\t8o6mFLmvkwf+7wJ+hyExxHpXC1Ylofuz+RwPX2Iisxw=\r\n\tResponse ticket hash:\t\t4Ve16TmWYd3ukhne7e50Tx6Sxf/NJFhKA2yvstCNinI=\r\n\r\nThis event is generated every time access is requested to a resource such as a computer or a Windows service.  The service name indicates the resource to which access was requested.\r\n\r\nThis event can be correlated with Windows logon events by comparing the Logon GUID fields in each event.  The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.\r\n\r\nPre-authentication types, ticket options, encryption types and result codes are defined in RFC 4120.","tags":["multiline"]}

Pipeline configuration:

PYTHON 
$pipeline =  
| from $source  
| eval customer=json_extract(_raw,"metadata.customer") 
| eval aideid=json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval _time = strptime(substr(_raw, 7, 29), "%Y-%m-%dT%H:%M:%S.%3N%z") 
| into $destination;

Pipeline 4: Extract the host name from the logs using regular expressions, parse a timestamp from the first 24 characters, and route the enriched records to the destination.

Sample event:

CODE 
26-Feb-2026 22:30:07.310 client 5.5.5.5#65784: query: vault.service.xxx.com IN A + (6.6.6.6) 
26-Feb-2026 22:30:07.310 client 6.6.6.6#3934: query: sxxx.net IN A + (7.7.7.7)

Pipeline configuration:

PYTHON 
$pipeline =  
| from $source  
| rex field=_raw /client\s(?P<host>\S+)#/ 
| eval _time = strptime(substr(_raw, 1, 24), "%d-%b-%Y %H:%M:%S.%3N") 
| into $destination;

Pipeline 5: Extract customer and device metadata from JSON, isolate the message content, remove the sourcetype field, parse a timestamp using the format day month date HH:MM:SS year, and then route the updated logs to the destination.

Sample event:

JSON 
{"metadata":{"proxy":{"port":"n/a","host":"n/a"},"aideid":"APP1","slt":{"pipeline":"p_zscalerzpa_syslogtcp","node":"bfdjkwendjw12","topic_id":"zscalerzpa-user-activity"},"source":{"port":43643,"host":"wdnjkwqn","type":"syslogtcp"},"timestamp":{"producer_process":"2026-02-27T18:15:34.603505486Z"},"customer":"Customer1"},"message":{"Policy":"Allow SIPA Connections","TimestampZENFirstRxConnector":"2026-02-27T18:15:33.058Z","ZENTotalBytesTxConnector":5953,"Server":"0","TimestampZENLastTxConnector":"2026-02-27T18:15:33.670Z","ClientLongitude":-11.1111,"Customer":"Company1","MicroTenantID":"0","ZENTotalBytesTxClient":09809,"ZENTotalBytesRxClient":4532,"ClientCity":"Miami","TimestampAppLearnStart":"","ClientCountryCode":"US","Idp":"0","ClientToClient":"0","sourcetype":"zscaler:zpa:UserActivity","ConnectionStatus":"close","ConnectorIP":"6.6.6.6","ConnectorPort":5345,"TimestampConnectionStart":"2026-02-27T18:15:32.939Z","ClientPublicIP":"7.7.7.7","AppMicroTenantID":"0","TimestampCARx":"2026-02-27T18:15:32.964Z","Application":"SIPA Domains","SessionID":"XX3XX2/XXXXXXXXXX","ZENBytesRxConnector":19575,"Connector":"SIPA_MSlogin_NTR_xxxxxx","ServicePort":443,"TimestampZENFirstRxClient":"2026-02-27T18:15:33.003Z","TimestampConnectionEnd":"2026-02-27T18:15:34.444Z","TimestampZENFirstTxClient":"2026-02-27T18:15:33.058Z","ServerIP":"1.1.1.1","TimestampConnectorZENSetupComplete":"2026-02-27T18:15:33.001Z","ClientZEN":"US-FL-7143","ZENBytesTxConnector":5953,"ZENTotalBytesRxConnector":19575,"PolicyProcessingTime":130,"TimestampZENFirstTxConnector":"2026-02-27T18:15:33.003Z","AppGroup":"SIPA","TimestampZENLastRxClient":"2026-02-27T18:15:33.668Z","ClientPrivateIP":"","ZENBytesRxClient":5953,"ServerSetupTime":11070,"IPProtocol":6,"DoubleEncryption":0,"Host":"login.xxx.com","TimestampZENLastTxClient":"2026-02-27T18:15:33.938Z","ZENBytesTxClient":19575,"ConnectionID":"XX3XX2/XXXXXXXXXX,XXX2XX+XXXXXXXXXX","Username":"abc789","ClientLatitude":11.1111,"ConnectorZEN":"US-IL-9407","TimestampCATx":"2026-02-27T18:15:32.939Z","LogTimestamp":"Fri Feb 27 18:15:34 2026","TimestampZENLastRxConnector":"2026-02-27T18:15:33.938Z","ServerPort":443,"InternalReason":"XXX_XX"}}

Pipeline configuration:

PYTHON 
$pipeline =  
| from $source  
| eval customer = json_extract(_raw,"metadata.customer") 
| eval aideid = json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval _raw = json_delete(_raw,"sourcetype") 
| eval ts = trim(json_extract(_raw, "LogTimestamp")) 
| eval _time = strptime(ts, "%a %b %d %H:%M:%S %Y") 
| fields -ts 
| into $destination;

The following table describes the amount of data per day that a single-instance Edge Processor can process, given the aforementioned test configurations and the specifications of the host machine. The "Maximum CPU utilization" and "Maximum memory usage" columns indicate the highest amounts of CPU and memory usage that were observed during performance testing.

Amazon EC2 instance type Maximum CPU utilization Maximum memory usage Throughput of incoming data
c6a.large 98% 45% 1.44 TB per day
c6a.xlarge 98% 21.4% 2.77 TB per day
c6a.2xlarge 86% 5.8% 5.36 TB per day
c5a.4xlarge 65% 8.2% 6.88 TB per day

This next table similarly describes the amount of data per day that a multi-instance Edge Processor can process:

Amazon EC2 instance type Number of Edge Processor instances Maximum CPU utilization Maximum memory usage Throughput of incoming data
c6a.2xlarge 5 95% 15% 27.6 TB per day

Sizing guidelines based on daily data ingestion volume

The table describes the following system requirements for processing a given amount of incoming data per day:

  • The number of Edge Processors that must be installed
  • The number of instances that each Edge Processor must be scaled out to
  • The amount of CPU and memory that the host machine of each Edge Processor instance must have
Note: These sizing guidelines are based on internal performance testing as described in Context for sizing guidelines. Results can vary for different environments and use cases, including differences in the number of source types handled and the amount of data being exported. As a best practice, take a conservative approach with the estimated results. Make sure to evaluate the resulting throughput of your Edge Processors and scale your deployment as necessary for your particular needs.
Data ingestion volume Edge Processors Instances CPU Memory
Less than 1.0 TB per day 1 1 2 vCPUs 4 GiB
1.0 to 2.5 TB per day 1 1 4 vCPUs 8 GiB
2.5 to 5.0 TB per day 1 1 8 vCPUs 16 GiB
5.0 to 6.5 TB per day 1 1 16 vCPUs 32 GiB
More than 6.5 TB per day 1 1 instance for every 5.0 TB of data 8 vCPUs 16 GiB

Mapping the sizing guidelines to a Kubernetes cluster

Size your Edge Processor deployment when running containers on Kubernetes using the Splunk-provided Helm chart.

The underlying Edge Processor binary is identical to the non-containerized version, making the throughput characteristics apply equally. The primary difference is how resources are expressed and managed. Instead of selecting a host machine of a given size, you configure CPU and memory limits on each pod. Kubernetes handles scheduling those pods onto appropriately sized nodes.

Kubernetes prerequisites

The following infrastructure requirements apply to the containerized Edge Processor deployment and have no equivalent in the non-containerized model:

  • Storage: A CSI-compatible storage provisioner is required for persistent volumes. On AWS, this means the Amazon EBS CSI driver must be installed and configured in your cluster. The Helm chart can optionally create a gp3-backed StorageClass for you by setting cloudProvider.aws.createStorageClass: true.

  • Cluster capacity: Ensure your node pool has enough total allocatable CPU and memory to satisfy not only your initial deployment.replicaCount × container.resources.limits, but also any additional replicas the HPA may add during scale-up.

  • Pod Disruption Budget: The chart deploys a PodDisruptionBudget with minAvailable: 3 by default, meaning at least three pods must remain running during voluntary disruptions such as node maintenance. Plan your replica count and node pool size accordingly.

Pod resource limits

In a containerized deployment model, the sizing unit is the pod. You can control how much CPU, memory, and storage each Edge Processor pod is allowed to consume by setting container.resources.requests and container.resources.limits in your chart's values.yaml file. The following table maps pod CPU and memory limits to the equivalent EC2 instance type from the performance analysis above, along with the expected throughput for a single Edge Processor instance. These equivalences hold because the C5 and C6 instance benchmarks are bottlenecked by vCPU count and available memory; therefore, a pod with matching limits will exhibit comparable throughput when scheduled on a node with sufficient capacity.

Pod CPU limit Pod memory limit Equivalent EC2 instance Throughput per pod
2 4 GiB c6a.large ~1.44 TB per day
4 8 GiB c6a.xlarge ~2.77 TB per day
8 16 GiB c6a.2xlarge ~5.36 TB per day
16 32 GiB c5a.4xlarge ~6.88 TB per day

The default limits provided in the Helm chart are 2 CPUs and 4 GiB of memory, which is equivalent to a c6a.large EC2 instance. Adjust these values based on your expected daily ingestion volume. To scale beyond a single pod's throughput, increase your chart's desired replica count. Consistent with the multi-instance results shown previously, total cluster throughput scales approximately linearly with replica count.

Node sizing

Each Kubernetes node must have enough allocatable CPU, memory, and storage to satisfy the combined resource requests of all pods scheduled onto it. As a general rule, provision nodes whose allocatable capacity is at least as large as the pod limits you have configured, with sufficient headroom for the Kubernetes system components and any other workloads sharing the node.

Persistent volume sizing

Each Edge Processor pod makes use of two persistent volumes:

  • Event queue (ep-event-queue): All events are written through a per-destination queue file before being exported. The queue drains continuously under normal conditions; however, unsent events will begin to accumulate if the ingestion rate exceeds the export rate or if the downstream destination is unavailable. If the queue reaches full capacity, the Edge Processor instance applies backpressure by no longer accepting new events, and upstream sources are left to handle the blockage on their own, such as by buffering locally or retrying continuously. Defaults to 15 GiB.

  • Instance identity (ep-instance-identity): Stores the pod's registered identity and configuration. Defaults to 1 MiB and should not require any manual adjustment.

Sizing the event queue

The event queue is designed to act as a buffer. Its size should depend on your total peak ingestion rate, the number of replicas sharing that traffic, as well as how long you want the queue to absorb incoming data before the pod must apply backpressure. To calculate the size in GiB, use the following formula:

CODE 
queue size (GiB) = 38.805 × peak ingestion rate (TB/day) × desired tolerance (hours) ÷ number of replicas

The constant 38.805 represents the conversion from TB/day to GiB/hour (i.e., 1 TB/day = 1,000,000,000,000 bytes/day ÷ 24 hours/day ÷ 1,073,741,824 bytes/GiB ≈ 38.805 GiB/hour). Additionally, the queue size is inversely proportional to the number of replicas because each pod handles a commensurate share of the traffic; thus, the per-pod queue size should decrease as the number of replicas increase. For example, a 3-replica deployment with a peak ingestion rate of 1.44 TB/day and a desired 1-hour tolerance requires approximately 18.6 GiB per pod.

Note: The event queue size is set at install time using persistence.eventQueue.size and cannot be resized without reprovisioning the persistent volume.
As ingestion volume grows over time, the preferred response is to increase the replica count rather than enlarging the per-pod queue. Scaling up replicas distributes the additional load across more pods and keeps individual queue sizes manageable, whereas simply growing the queue only delays the point at which backpressure is applied without increasing actual throughput capacity. Instead, reserve queue size increases for situations where you need a longer tolerance window. For example, to survive a longer downstream outage. Not as a substitute for horizontal scaling.

Horizontal scaling and scale-down behavior

The Helm chart includes a Horizontal Pod Autoscaler that scales the number of Edge Processor pods based on CPU and memory utilization. Scale-downs are disabled by default, however, since terminating a pod before its event queue has fully drained orphans any queued events on the persistent volume. Therefore, until queue-depth-aware scaling is available, it is not considered safe to automate the removal/deletion of pods.