Sizing guidelines for Edge Processors

Scale your Edge Processors to provide the necessary support for your incoming data.

The system requirements for your deployment of the Edge Processor solution can vary depending on the volume of data being processed and the complexity of the processing operations involved. In order to support higher volumes of incoming data, or more complex data transformations such as field extraction and normalization, you might need to install your Edge Processors on host machines with higher specifications or scale out your Edge Processors by adding more instances.

To determine how to scale your Edge Processor deployment, start by estimating the amount of incoming data that your Edge Processors will ingest and process per day. Then, refer to the table in the Sizing guidelines based on daily data ingestion volume section on this page.

Note: Be aware that the maximum number of Edge Processor instances that can be supported depends on the capabilities of your Data Management control plane. See Setup prerequisites for more information.

The sizing guidelines were determined based on internal performance testing. For information about the configurations and specifications used in the performance tests, see Context for sizing guidelines.

Sizing guidelines based on daily data ingestion volume

The table describes the following system requirements for processing a given amount of incoming data per day:

  • The number of Edge Processors that must be installed
  • The number of instances that each Edge Processor must be scaled out to
  • The amount of CPU and memory that the host machine of each Edge Processor instance must have
Note: These sizing guidelines are based on internal performance testing as described in Context for sizing guidelines. Results can vary for different environments and use cases, including differences in the number of source types handled and the amount of data being exported. As a best practice, take a conservative approach with the estimated results. Make sure to evaluate the resulting throughput of your Edge Processors and scale your deployment as necessary for your particular needs.
Data ingestion volume Edge Processors Instances CPU Memory
Less than 1.0 TB per day 1 1 2 vCPUs 4 GiB
1.0 to 2.5 TB per day 1 1 4 vCPUs 8 GiB
2.5 to 5.0 TB per day 1 1 8 vCPUs 16 GiB
5.0 to 6.5 TB per day 1 1 16 vCPUs 32 GiB
More than 6.5 TB per day 1 1 instance for every 5.0 TB of data 8 vCPUs 16 GiB

Context for sizing guidelines

The guidelines shown in the previous section were determined based on internal performance testing using the following configurations:

  • The Edge Processor instances were installed on Amazon EC2 C5a and C6a instances. For information about the specifications, see the following pages on the Amazon Web Services website:

  • Each Edge Processor was processing data from 50 applied pipelines, and each pipeline was partitioned on a single source.

  • The Splunk-to-Splunk (S2S) protocol was used for data transmission. The data was sent from a heavy forwarder to the Edge Processor, and then from the Edge Processor to a Splunk platform deployment through an S2S connection.
    • The test setup consisted of 80 to 500 heavy forwarder instances.

    • The heavy forwarders handled the line-breaking of the incoming events, so the data processing workload of the Edge Processors did not include line-breaking.

  • The incoming data consisted of Palo Alto Network logs and Windows Event logs with an average event size ranging from 70 bytes to 8.6 kilobytes.

  • The data was processed by pipelines that enriched the events with additional information by parsing and extracting fields from the events.

For more information about the data and pipeline configuration used in the performance tests, see the expandable sections that follow. Each section shows a representative example of an event and the configuration of the pipeline that was used during performance testing.

Pipeline 1: Extract customer and device metadata from Palo Alto Networks syslog messages, classify the messages by log type, and enrich the logs with host name values before routing them to the destination.

Sample event:

JSON
{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:27.087874108Z"}},"message":"<14>1 2026-04-01T15:24:27+00:00 hostname - - - - 1,2026/04/01 15:24:26,010108001430,TRAFFIC,end,2561,2026/04/01 15:24:26,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXX,,,ssl,vsys1,internal,external,ethernet1/1.1,ethernet1/2.2,log-forwarding-main,2026/04/01 15:24:26,71538790,1,65406,23775,443,0x40047a,tcp,allow,24074,8570,15504,44,2026/04/01 14:54:17,1808,computer-and-internet-info,,72638213781273912,0x8000000000000000,1.1.1.1,United States,,22,22,aged-out,5267,53,52,0,vsys1,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,sdnwjq-sdsa-4533-ab21-adnandjk88,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:27.085+00:00,,,encrypted-tunnel,networking,browser-based,4,\"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use\",,ssl,no,no,0"} 

{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:28.511259452Z"}},"message":"<14>1 2026-04-01T15:24:28+00:00 hostname - - - - 1,2026/04/01 15:24:08,010701010641,TRAFFIC,end,2817,2026/04/01 15:24:08,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXXAware - Managed DNS,,,dns-base,vsys1,l3-internal,l3-external,ae3.15,ethernet2/28.10,log-forwarding-main-dns-only,2026/04/01 15:24:27,2793178772,1,37709,53,7102,53,0x400019,udp,allow,293,113,180,2,2026/04/01 15:23:37,0,any,,sdvdsvd,0x8000000000000000,1.1.1.1,United States,,1,1,aged-out,5267,53,50,0,vsys1,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,c6db958a-f3e9-4264-9e87-39f37973446b,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:28.200+00:00,,,infrastructure,networking,network-protocol,3,\"used-by-malware,has-known-vulnerability,pervasive-use\",dns,dns-base,no,no,0,NonProxyTraffic,"} 

{"metadata":{"proxy":{"address":"1.1.1.1"},"aideid":"XXXX","source":{"type":"syslog"},"customer":"ABC","timestamp":{"producer_process":"2026-04-01T15:24:28.510794234Z"}},"message":"<14>1 2026-04-01T15:24:28+00:00 hostname - - - - 1,2026/04/01 15:24:08,010701010641,TRAFFIC,end,2817,2026/04/01 15:24:08,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,XXXXX,,,ssl,vsys3,internal,external,ethernet1/25.88,ethernet1/26.96,log-forwarding-main,2026/04/01 15:24:27,2958683727,1,58877,443,35059,443,0x40047a,tcp,allow,9346,3759,5587,27,2026/04/01 15:23:48,16,urlcat_RITM2954011,,7603808268918733999,0x8000000000000000,1.1.1.1,XXX,,14,13,tcp-fin,5267,4553,4556,0,cit,hostname,from-policy,,,0,,0,,N/A,0,0,0,0,a132b316-bdc6-403a-9153-e1575e552284,0,0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,2026-04-01T15:24:28.200+00:00,,,encrypted-tunnel,networking,browser-based,4,\"used-by-malware,able-to-transfer-file,has-known-vulnerability,tunnel-other-application,pervasive-use\",,ssl,no,no,0,NonProxyTraffic,"}

Pipeline configuration:

PYTHON
$pipeline =  
| from $source  
| eval customer=json_extract(_raw,"metadata.customer") 
| eval aideid = json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval sourcetype = case (like(_raw, "%TRAFFIC%"),"pan:traffic", like(_raw, "%THREAT%"),"pan:threat", like(_raw, "%SYSTEM%"),"pan:system" ,like(_raw, "%CONFIG%"), "pan:config",like(_raw, "%CORRELATION%"), "pan:correlation",like(_raw, "%HIPMATCH%"), "pan:hipmatch",like(_raw, "%GLOBALPROTECT%"), "pan:globalprotect",like(_raw, "%USERID%"), "pan:userid",true,"pan:log") 
| rex field=_raw /^<\d+>\d+\s+\S+\s+(?P<host>\S+)/ 
| into $destination;

Pipeline 2: Extract Windows Event logs from JSON, parse the XML-formatted event messages to identify the channel, and enrich the logs with host name values and a channel-based source identifier before routing them to the destination.

Sample event:

JSON
{"agent":{"name":"server1"},"process":{"name":"-","pid":0,"executable":"-"},"winlog":{"computer_name":"comp1","process":{"pid":1128,"thread":{"id":15412}},"keywords":["Audit Success"],"logon":{"id":"1X11X111X","type":"Network"},"channel":"Security","event_data":{"LogonGuid":"{00000000-0000-0000-0000-000000000000}","TargetOutboundDomainName":"-","VirtualAccount":"%%1843","LogonType":"3","TransmittedServices":"-","SubjectLogonId":"0x0","TargetOutboundUserName":"-","KeyLength":"128","LmPackageName":"NTLM V2","TargetLogonId":"1X11X111X","RestrictedAdminMode":"-","TargetLinkedLogonId":"0x0","SubjectUserName":"-","ElevatedToken":"%%1843","SubjectDomainName":"-","TargetUserName":"abc123","ImpersonationLevel":"%%1833","TargetDomainName":"MS","LogonProcessName":"NtLmSsp ","SubjectUserSid":"S-1-0-0","TargetUserSid":"X-1-1-11-111111111-1111111111-1111111111-1111111","AuthenticationPackageName":"NTLM"},"opcode":"Info","version":2,"record_id":8525424013,"task":"Logon","event_id":"4624","provider_guid":"{54849625-5478-4994-a5ba-3e3b0328c30d}","activity_id":"{11111111-XX11-1111-1111-11111XX1XX11}","api":"wineventlog","provider_name":"Microsoft-Windows-Security-Auditing"},"log":{"level":"information"},"@metadata":{"beat":"winlogbeat","type":"_doc","version":"8.8.2"},"event_original":"<Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-Security-Auditing' Guid='{54849625-5478-4994-a5ba-3e3b0328c30d}'/><EventID>4624</EventID><Version>2</Version><Level>0</Level><Task>12544</Task><Opcode>0</Opcode><Keywords>0x8020000000000000</Keywords><TimeCreated SystemTime='2026-03-05T20:55:44.999795600Z'/><EventRecordID>8525424013</EventRecordID><Correlation ActivityID='{11111111-XX11-1111-1111-11111XX1XX11}'/><Execution ProcessID='1128' ThreadID='15412'/><Channel>Security</Channel><Computer>comp1</Computer><Security/></System><EventData><Data Name='SubjectUserSid'>S-1-0-0</Data><Data Name='SubjectUserName'>-</Data><Data Name='SubjectDomainName'>-</Data><Data Name='SubjectLogonId'>0x0</Data><Data Name='TargetUserSid'>X-1-1-11-111111111-1111111111-1111111111-1111111</Data><Data Name='TargetUserName'>abc123</Data><Data Name='TargetDomainName'>MS</Data><Data Name='TargetLogonId'>1X11X111X</Data><Data Name='LogonType'>3</Data><Data Name='LogonProcessName'>NtLmSsp </Data><Data Name='AuthenticationPackageName'>NTLM</Data><Data Name='WorkstationName'>WORKSTATION1</Data><Data Name='LogonGuid'>{00000000-0000-0000-0000-000000000000}</Data><Data Name='TransmittedServices'>-</Data><Data Name='LmPackageName'>NTLM V2</Data><Data Name='KeyLength'>128</Data><Data Name='ProcessId'>0x0</Data><Data Name='ProcessName'>-</Data><Data Name='IpAddress'>1.1.1.1</Data><Data Name='IpPort'>50044</Data><Data Name='ImpersonationLevel'>%%1833</Data><Data Name='RestrictedAdminMode'>-</Data><Data Name='TargetOutboundUserName'>-</Data><Data Name='TargetOutboundDomainName'>-</Data><Data Name='VirtualAccount'>%%1843</Data><Data Name='TargetLinkedLogonId'>0x0</Data><Data Name='ElevatedToken'>%%1843</Data></EventData><RenderingInfo Culture='en-US'><Message>An account was successfully logged on.\r\n\r\nSubject:\r\n\tSecurity ID:\t\tS-1-0-0\r\n\tAccount Name:\t\t-\r\n\tAccount Domain:\t\t-\r\n\tLogon ID:\t\t0x0\r\n\r\nLogon Information:\r\n\tLogon Type:\t\t3\r\n\tRestricted Admin Mode:\t-\r\n\tVirtual Account:\t\tNo\r\n\tElevated Token:\t\tNo\r\n\r\nImpersonation Level:\t\tImpersonation\r\n\r\nNew Logon:\r\n\tSecurity ID:\t\tX-1-1-11-111111111-1111111111-1111111111-1111111\r\n\tAccount Name:\t\tabc123\r\n\tAccount Domain:\t\tMS\r\n\tLogon ID:\t\t1X11X111X\r\n\tLinked Logon ID:\t\t0x0\r\n\tNetwork Account Name:\t-\r\n\tNetwork Account Domain:\t-\r\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\r\n\r\nProcess Information:\r\n\tProcess ID:\t\t0x0\r\n\tProcess Name:\t\t-\r\n\r\nNetwork Information:\r\n\tWorkstation Name:\tWORKSTATION1\r\n\tSource Network Address:\t1.1.1.1\r\n\tSource Port:\t\t50044\r\n\r\nDetailed Authentication Information:\r\n\tLogon Process:\t\tNtLmSsp \r\n\tAuthentication Package:\tNTLM\r\n\tTransited Services:\t-\r\n\tPackage Name (NTLM only):\tNTLM V2\r\n\tKey Length:\t\t128\r\n\r\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\r\n\r\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\r\n\r\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\r\n\r\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\r\n\r\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\r\n\r\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\r\n\r\nThe authentication information fields provide detailed information about this specific logon request.\r\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\r\n\t- Transited services indicate which intermediate services have participated in this logon request.\r\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\r\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.</Message><Level>Information</Level><Task>Logon</Task><Opcode>Info</Opcode><Channel>Security</Channel><Provider>Microsoft Windows security auditing.</Provider><Keywords><Keyword>Audit Success</Keyword></Keywords></RenderingInfo></Event>","source":{"port":50044,"ip":"1.1.1.1","domain":"WORKSTATION1"},"message":"An account was successfully logged on.\n\nSubject:\n\tSecurity ID:\t\tS-1-0-0\n\tAccount Name:\t\t-\n\tAccount Domain:\t\t-\n\tLogon ID:\t\t0x0\n\nLogon Information:\n\tLogon Type:\t\t3\n\tRestricted Admin Mode:\t-\n\tVirtual Account:\t\tNo\n\tElevated Token:\t\tNo\n\nImpersonation Level:\t\tImpersonation\n\nNew Logon:\n\tSecurity ID:\t\tX-1-1-11-111111111-1111111111-1111111111-1111111\n\tAccount Name:\t\tabc123\n\tAccount Domain:\t\tMS\n\tLogon ID:\t\t1X11X111X\n\tLinked Logon ID:\t\t0x0\n\tNetwork Account Name:\t-\n\tNetwork Account Domain:\t-\n\tLogon GUID:\t\t{00000000-0000-0000-0000-000000000000}\n\nProcess Information:\n\tProcess ID:\t\t0x0\n\tProcess Name:\t\t-\n\nNetwork Information:\n\tWorkstation Name:\tWORKSTATION1\n\tSource Network Address:\t1.1.1.1\n\tSource Port:\t\t50044\n\nDetailed Authentication Information:\n\tLogon Process:\t\tNtLmSsp \n\tAuthentication Package:\tNTLM\n\tTransited Services:\t-\n\tPackage Name (NTLM only):\tNTLM V2\n\tKey Length:\t\t128\n\nThis event is generated when a logon session is created. It is generated on the computer that was accessed.\n\nThe subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.\n\nThe logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).\n\nThe New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.\n\nThe network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.\n\nThe impersonation level field indicates the extent to which a process in the logon session can impersonate.\n\nThe authentication information fields provide detailed information about this specific logon request.\n\t- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.\n\t- Transited services indicate which intermediate services have participated in this logon request.\n\t- Package name indicates which sub-protocol was used among the NTLM protocols.\n\t- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.","@timestamp":"2026-03-05T20:55:44.999Z","related":{"ip":"1.1.1.1","user":"abc123"},"event":{"code":"4624","provider":"Microsoft-Windows-Security-Auditing","created":"2026-03-05T20:55:45.707Z","kind":"event","module":"security","action":"logged-in","type":["start"],"category":["authentication"],"outcome":"success"},"fields":{"log_topic":"microsoft-windows"},"user":{"domain":"MS","name":"abc123","id":"X-1-1-11-111111111-1111111111-1111111111-1111111"}}

Pipeline configuration:

PYTHON
$pipeline = | from $source  
| eval host=json_extract(_raw,"winlog.computer_name") 
| eval _raw=json_extract(_raw,"event_original") 
| rex field=_raw /<Channel>(?P<win_channel>[^<]+)<\/Channel>/ 
| eval source="XmlWinEventLog:"+win_channel 
| into $destination;

Pipeline 3: Extract customer and device metadata from JSON, isolate the message content, parse an ISO 8601 timestamp from it, and route the enriched records to the destination.

Sample event:

JSON
{"metadata":{"proxy":{"address":"2.2.2.2","port":"500"},"aideid":"APP1","source":{"address":"src1","port":"00000","type":"syslog"},"timestamp":{"producer_process":"2026-02-17T18:32:46.110886807Z"},"customer":"Customer1"},"message":"<13>1 2026-02-17T12:09:37.000-06:00 comp1 InTrust - 4769 [InTrust.Predefined@3973 RecordKey=\"1427165792\" EventType=\"8\" UserName=\"\" UserDomain=\"\" SourceName=\"Microsoft-Windows-Security-Auditing\" ComputerName=\"comp1\" NumericCategory=\"14337\" Category=\"Kerberos Service Ticket Operations\" EventID=\"4769\" GatheringEventLog=\"Security\" GatheringComputer=\"XXXXXXXXXXXXXXX\" GatheringDomain=\"MS\" PlatformID=\"500\" VersionMajor=\"10\" VersionMinor=\"0\" ComputerType=\"93489320\" DatasourceType=\"{X9X5X7X2-5X01-41X7-9X36-X562XXXXXA9}\" Environment=\"9X442XXX-XXX2-4X79-9013-053XX225XXX0\"][InTrust.Named@3973 Account_Name=\"XXX.XXX.COM\" Account_Domain=\"XXX.XXX.COM\" Service_Name=\"XXXXXXXXXXXXX\" Service_ID=\"MS\\\\DXBFSDJKFBESJK\" Source_Network_Address=\"::ffff:3.3.3.3\" Source_Port=\"63888\" Ticket_Options=\"0x40800000\" Failure_Code=\"0x0\" Ticket_Encryption_Type=\"0x12\" Transited_Services=\"-\" Failure_Reason=\"Successful\" Who=\"XXX.XXX.COM\" WhoDomain=\"XXX.XXX.COM\" What=\"Kerberos Service Ticket Operations\" Where=\"comp1\" Severity=\"High\" Where_From=\"::ffff:3.3.3.3\" Result=\"Successful\"] A Kerberos service ticket was requested.\r\n\r\nAccount Information:\r\n\tAccount Name:\t\tXXX.XXX.COM\r\n\tAccount Domain:\t\tXXX.XXX.COM\r\n\tLogon GUID:\t\t{8X932478-X75X-8XXX-3X5X-796X7276117X}\r\n\tMSDS-SupportedEncryptionTypes:\tN/A\r\n\tAvailable Keys:\tN/A\r\n\r\nService Information:\r\n\tService Name:\t\tDXBFSDJKFBESJK\r\n\tService ID:\t\tMS\\DXBFSDJKFBESJK\r\n\tMSDS-SupportedEncryptionTypes:\t0x1F (DES, RC4, AES128-SHA96, AES256-SHA96)\r\n\tAvailable Keys:\tAES-SHA1, RC4\r\n\r\nDomain Controller Information:\r\n\tMSDS-SupportedEncryptionTypes:\t0x1F (DES, RC4, AES128-SHA96, AES256-SHA96)\r\n\tAvailable Keys:\tAES-SHA1, RC4\r\n\r\nNetwork Information:\r\n\tClient Address:\t\t::ffff:3.3.3.3\r\n\tClient Port:\t\t63888\r\n\tAdvertized Etypes:\t\n\t\tAES256-CTS-HMAC-SHA1-96\n\t\tAES128-CTS-HMAC-SHA1-96\n\t\tRC4-HMAC-NT\r\n\r\nAdditional Information:\r\n\tTicket Options:\t\t0x40800000\r\n\tTicket Encryption Type:\t0x12\r\n\tSession Encryption Type:\t0x12\r\n\tFailure Code:\t\t0x0\r\n\tTransited Services:\t-\r\n\r\nTicket information\r\n\tRequest ticket hash:\t\t8o6mFLmvkwf+7wJ+hyExxHpXC1Ylofuz+RwPX2Iisxw=\r\n\tResponse ticket hash:\t\t4Ve16TmWYd3ukhne7e50Tx6Sxf/NJFhKA2yvstCNinI=\r\n\r\nThis event is generated every time access is requested to a resource such as a computer or a Windows service.  The service name indicates the resource to which access was requested.\r\n\r\nThis event can be correlated with Windows logon events by comparing the Logon GUID fields in each event.  The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.\r\n\r\nPre-authentication types, ticket options, encryption types and result codes are defined in RFC 4120.","tags":["multiline"]}

Pipeline configuration:

PYTHON
$pipeline =  
| from $source  
| eval customer=json_extract(_raw,"metadata.customer") 
| eval aideid=json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval _time = strptime(substr(_raw, 7, 29), "%Y-%m-%dT%H:%M:%S.%3N%z") 
| into $destination;

Pipeline 4: Extract the host name from the logs using regular expressions, parse a timestamp from the first 24 characters, and route the enriched records to the destination.

Sample event:

CODE
26-Feb-2026 22:30:07.310 client 5.5.5.5#65784: query: vault.service.xxx.com IN A + (6.6.6.6) 
26-Feb-2026 22:30:07.310 client 6.6.6.6#3934: query: sxxx.net IN A + (7.7.7.7)

Pipeline configuration:

PYTHON
$pipeline =  
| from $source  
| rex field=_raw /client\s(?P<host>\S+)#/ 
| eval _time = strptime(substr(_raw, 1, 24), "%d-%b-%Y %H:%M:%S.%3N") 
| into $destination;

Pipeline 5: Extract customer and device metadata from JSON, isolate the message content, remove the sourcetype field, parse a timestamp using the format day month date HH:MM:SS year, and then route the updated logs to the destination.

Sample event:

JSON
{"metadata":{"proxy":{"port":"n/a","host":"n/a"},"aideid":"APP1","slt":{"pipeline":"p_zscalerzpa_syslogtcp","node":"bfdjkwendjw12","topic_id":"zscalerzpa-user-activity"},"source":{"port":43643,"host":"wdnjkwqn","type":"syslogtcp"},"timestamp":{"producer_process":"2026-02-27T18:15:34.603505486Z"},"customer":"Customer1"},"message":{"Policy":"Allow SIPA Connections","TimestampZENFirstRxConnector":"2026-02-27T18:15:33.058Z","ZENTotalBytesTxConnector":5953,"Server":"0","TimestampZENLastTxConnector":"2026-02-27T18:15:33.670Z","ClientLongitude":-11.1111,"Customer":"Company1","MicroTenantID":"0","ZENTotalBytesTxClient":09809,"ZENTotalBytesRxClient":4532,"ClientCity":"Miami","TimestampAppLearnStart":"","ClientCountryCode":"US","Idp":"0","ClientToClient":"0","sourcetype":"zscaler:zpa:UserActivity","ConnectionStatus":"close","ConnectorIP":"6.6.6.6","ConnectorPort":5345,"TimestampConnectionStart":"2026-02-27T18:15:32.939Z","ClientPublicIP":"7.7.7.7","AppMicroTenantID":"0","TimestampCARx":"2026-02-27T18:15:32.964Z","Application":"SIPA Domains","SessionID":"XX3XX2/XXXXXXXXXX","ZENBytesRxConnector":19575,"Connector":"SIPA_MSlogin_NTR_xxxxxx","ServicePort":443,"TimestampZENFirstRxClient":"2026-02-27T18:15:33.003Z","TimestampConnectionEnd":"2026-02-27T18:15:34.444Z","TimestampZENFirstTxClient":"2026-02-27T18:15:33.058Z","ServerIP":"1.1.1.1","TimestampConnectorZENSetupComplete":"2026-02-27T18:15:33.001Z","ClientZEN":"US-FL-7143","ZENBytesTxConnector":5953,"ZENTotalBytesRxConnector":19575,"PolicyProcessingTime":130,"TimestampZENFirstTxConnector":"2026-02-27T18:15:33.003Z","AppGroup":"SIPA","TimestampZENLastRxClient":"2026-02-27T18:15:33.668Z","ClientPrivateIP":"","ZENBytesRxClient":5953,"ServerSetupTime":11070,"IPProtocol":6,"DoubleEncryption":0,"Host":"login.xxx.com","TimestampZENLastTxClient":"2026-02-27T18:15:33.938Z","ZENBytesTxClient":19575,"ConnectionID":"XX3XX2/XXXXXXXXXX,XXX2XX+XXXXXXXXXX","Username":"abc789","ClientLatitude":11.1111,"ConnectorZEN":"US-IL-9407","TimestampCATx":"2026-02-27T18:15:32.939Z","LogTimestamp":"Fri Feb 27 18:15:34 2026","TimestampZENLastRxConnector":"2026-02-27T18:15:33.938Z","ServerPort":443,"InternalReason":"XXX_XX"}}

Pipeline configuration:

PYTHON
$pipeline =  
| from $source  
| eval customer = json_extract(_raw,"metadata.customer") 
| eval aideid = json_extract(_raw,"metadata.aideid") 
| eval _raw = json_extract(_raw,"message") 
| eval _raw = json_delete(_raw,"sourcetype") 
| eval ts = trim(json_extract(_raw, "LogTimestamp")) 
| eval _time = strptime(ts, "%a %b %d %H:%M:%S %Y") 
| fields -ts 
| into $destination;

The following table describes the amount of data per day that a single-instance Edge Processor can process, given the aforementioned test configurations and the specifications of the host machine. The "Maximum CPU utilization" and "Maximum memory usage" columns indicate the highest amounts of CPU and memory usage that were observed during performance testing.

Amazon EC2 instance type Maximum CPU utilization Maximum memory usage Throughput of incoming data
c6a.large 98% 45% 1.44 TB per day
c6a.xlarge 98% 21.4% 2.77 TB per day
c6a.2xlarge 86% 5.8% 5.36 TB per day
c5a.4xlarge 65% 8.2% 6.88 TB per day

This next table similarly describes the amount of data per day that a multi-instance Edge Processor can process:

Amazon EC2 instance type Number of Edge Processor instances Maximum CPU utilization Maximum memory usage Throughput of incoming data
c6a.2xlarge 5 95% 15% 27.6 TB per day