Convert data in the “_raw” field to OCSF format

Use the ocsf SPL2 command in an Ingest Processor pipeline to convert data from a raw text format to the Open Cybersecurity Schema Framework (OCSF) format.

If the data that you want to convert to OCSF format is stored in an event field called _raw, and you want to store the OCSF-formatted results in the _raw field, then use the ocsf SPL2 command in your pipeline.

Note: As part of the data conversion process, the ocsf command prefixes the specified source type with ocsf: and then stores the updated source type in the sourcetype field. This change ensures that your data is associated with a source type that is supported by the OCSF-CIM Add-on for Splunk and Splunk Enterprise Security.
  1. On the Pipelines page, select New pipeline. Follow the on-screen instructions to define a partition, optionally enter sample data, and select a data destination.
    After you complete the on-screen instructions, the pipeline editor displays the SPL2 statement for your pipeline.
  2. Select the plus icon (This image shows an icon of a plus sign.) in the Actions section of the pipeline builder, and then select Convert _raw to OCSF format.
  3. Specify the source type that you want the data to be parsed as during conversion. You can set the Source type option to a static value, or to a value that is read dynamically from your data. Do one of the following:
    ScenarioHow to specify the source type
    You want to use a static value as the source type.Set the drop-down list to one of the supported source types.
    You want to use the exact values from an event field as the source type.

    Set the drop-down list to Field, and then specify the name of the event field.

    You can also set the drop-down list to Field and then leave the field name unspecified. In this case, the Ingest Processor defaults to using the sourcetype field.

    You want to use the result of an SPL2 expression as the source type.

    For example, if the source type is part of a field value that includes other text, you can use an SPL2 expression to extract the source type.

    Set the drop-down list to Expression and then enter an SPL2 expression that resolves to the source type.
  4. (Optional) Configure the following advanced options to include additional information in the OCSF-formatted output:
    • To include a raw_data attribute that contains a copy of the original data, select Include original raw data.

    • To include descriptive labels for ID values, select Include sibling strings for enum attributes. For more information about this configuration option, see Including sibling strings for enum attributes.

    • To include an observables array that summarizes the attributes that contain security observables, select Include observables. For more information about this configuration option, see Including observables.

  5. Select Apply.
    The pipeline editor adds an import statement and an ocsf command to your pipeline. The import statement imports the ocsf command into the pipeline so that the command is available for use.
  6. (Optional) To filter your data for failed OCSF conversions and send those results to a different destination than the successfully converted data, do the following:
    1. Select the plus icon (This image shows an icon of a plus sign.) in the Actions section of the pipeline builder, and then select Route a subset of data.
    2. Configure the options in the Route data dialog box as follows:
      Option nameEnter or select the following

      Field

      Set the drop-down list to Expression and then enter the following:
      json_extract(_raw, "class_uid")

      Action

      Include

      Operator

      = equals

      Value

      6008

      Match case

      This option is not used when matching numbers, so you don't need to do anything with it.

    3. Select Apply.
      The pipeline editor updates the import statement to include the route command, and adds a route command to your pipeline.
    4. In the Actions section of the pipeline builder, select Send data to $destination2. Select the destination that you want to send the failed OCSF conversions to, and then select Apply.
    For information about how the Ingest Processor handles failed OCSF conversions, see Fallback behavior for failed conversions.
  7. Save your pipeline, and then apply it to your Ingest Processor as needed. For more information, see Apply a pipeline.
You now have a pipeline that converts the data in the _raw field from a raw text format into the OCSF format. The pipeline also prefixes the specified source type with ocsf: and then stores the updated source type in the sourcetype field.

Example: Use the ocsf command to convert data

For example, assume that the pipeline receives the following event:
_rawsource_device
<166>Oct 06 2021 12:56:34 10.160.0.10 : %ASA-6-611101: User authentication succeeded: IP address: 10.160.39.123, Uname: admincisco:asa
You can process that event using the following pipeline:
import ocsf from /splunk.ingest.commands

$pipeline = | from $source | ocsf sourcetype=source_device include_raw=true | into $destination;
This pipeline does the following:
  • Parses the data from the _raw field based on the source type indicated in the source_device field.

  • Converts the data into OCSF format.

  • Includes a copy of the original data in the raw_data attribute in the converted data.

  • Overwrites the _raw field with the converted data.

  • Adds a sourcetype field that contains the value from the source_device field prefixed with ocsf:.

The resulting event looks like this:
_rawsource_devicesourcetype
{ 
    category_uid: 3,
    metadata: { 
        uid: "3814d6ec4a9fe689ef0010c2c5823b00",
        product: { 
            name: "ASA",
            vendor_name: "Cisco"
        },
        log_name: "Syslog",
        event_code: "611101",
        profiles: [ 
            "host"
        ],
        original_time: "Oct 06 2021 12:56:34",
        version: "1.5.0"
    },
    session: {
        is_vpn: true
    },
    src_endpoint: {
        ip: "10.160.39.123"
    },
    message: "User authentication succeeded: IP address: 10.160.39.123, Uname: admin",
    unmapped: {
        level: "6",
        facility: 20
    },
    status_id: 1,
    service: {
        name: "ASA"
    },
    activity_id: 1,
    class_uid: 3002,
    dst_endpoint: {
        ip: "10.160.0.10"
    },
    severity_id: 1,
    time: 1633524994000,
    device: {
        type_id: 9,
        ip: "10.160.0.10"
    },
    raw_data: "<166>Oct 06 2021 12:56:34 10.160.0.10 : %ASA-6-611101: User authentication succeeded: IP address: 10.160.39.123, Uname: admin",
    user: {
        name: "admin"
    },
    type_uid: 300201
}

cisco:asa

ocsf:cisco:asa

For more information, see the following pages: