Use the Decrypt command to decrypt data in the Ingest Processor solution

The Ingest Processor solution allows you to send encrypted data through your pipelines, and decrypt it before it reaches its destination. That way, you do not have to decrypt your data before processing it in Ingest Processor pipelines. To decrypt your data, apply the Decrypt command to your pipelines.

The Decrypt command is an SPL2 command that requires a private key, which must be stored in a lookup table. The Decrypt command has four required fields: the field to decrypt, the name of the lookup table that your private key is stored under, the specific lookup field name within your lookup table where your private key is stored, and the name of the field where the decrypted value will be outputted.

The Ingest Processor itself does not encrypt data, so your data must already be encrypted before it enters the pipeline.

Prerequisites

  • The data must already be encrypted using the RSA algorithm and PKCS1#1 v1.5 padding.
  • The private key must be stored in a lookup table. If an invalid private key is used, the decrypt command will return a placeholder NIL string. For more information on using lookups for the Ingest Processor solution, see

    Enrich Data with Lookups using an Ingest Processor. One column in the lookup table must have the exact title private_key. See the following example of a lookup table CSV file:

private_key, device_id MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCO0wIiso9DBXCIR82prtAf+TnN1aKvZ7oC7rSpaJSIoAI2ijmJh/q+5fhn7Ku7ktBXvM5fw+UcknVBJJewz9MVb3OzvL2DFUydq7dpU+1hEWkNH6skSFVX, 3F2504E0
Note:

Please note that RSA decryption is a resource-intensive operation. As a result, you may observe a decrease in throughput depending on the amount of load being sent, especially when compared to a regular pipeline without decryption enabled.

This behavior is expected due to the additional computational overhead introduced by RSA decryption.

Use the decrypt command

To decrypt your data, see the following steps:

  1. Navigate to the Pipelines page.

  2. Select the Actions icon (Image of the Actions icon) in the row that lists your pipeline, and then select Edit. This will take you to the pipeline editor for that pipeline.

  3. Select the plus icon (This image shows an icon of a plus sign.) in the Actions area of the pipeline builder and select Decrypt field using lookup.

  4. In the menu, provide the name of the Lookup table that your private key is stored under, the name of the specific Lookup match field, the Field to decrypt and the Decrypted output field where you will store your output.

  5. Select Apply.

Example: Use the Decrypt command to decrypt data

Consider a scenario where the pipeline receives the following event with an encrypted SerialNumber field:
{"device_id": "3F2504E0", "device_type": "router", "serial_number": "U2FsdGVkX1+9K2pQ7c3gX0yH4mN5v6wR1aB8zLpDqFjEwXcVxYtZsGhIuO0P1r2sY"}
The private key to decrypt the encrypted field is stored in the cproc-decrypt.csv CSV lookup table. It could also be stored in a KV Store equivalent:
cproc-decrypt.csv (or KV Store equivalent)

private_key, device_id

MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCO0wIiso9DBXCIR82prtAf+TnN1aKvZ7oC7rSpaJSIoAI2ijmJh/q+5fhn7Ku7ktBXvM5fw+UcknVBJJewz9MVb3OzvL2DFUydq7dpU+1hEWkNH6skSFVX, 3F2504E0
See the following example code to decrypt your encrypted field for this scenario:
decrypt encrypted_payload='serial_number' keystore='cproc-decrypt.csv' key_config='device_id' decrypted_output_field='decrypted_field_output'
Where:

  • encrypted_payload is the encrypted data field to be decrypted

  • keystore is the lookup table name that contains the private key to decrypt the encrypted field

  • key_config is the specific lookup field name within your lookup table where your private key is stored

  • decrypted_output_field is the name of the field where the decrypted value will be stored