Process your search results

Process, manipulate, and perform calculations on your search results by adding commands and functions to your SPL2 search.

You can process, manipulate, and perform calculations on your search results by adding commands and functions to your SPL2 search. If you’re using the from command to search your data, then you can also use the optional clauses supported in this command to work with your data.

SPL2 provides a broad and varied range of commands and functions for completing data processing actions on your search results. The following are some examples of data processing actions that use 2 of the most commonly used SPL2 commands: the eval command and the stats command.

For comprehensive information about all of the available SPL2 commands and functions, see the SPL2 Search Reference. In particular, for information about the eval and stats commands used in the examples that follow, see the eval and stats pages in the SPL2 Search Reference.

Evaluate string expressions

You can evaluate string expressions by using the eval command. For example, you can format all the string values in a given field into lowercase.

Note: For information about SPL2 expressions, see Types of expressions.

To add the eval command to your search, enter a pipe ( | ) character followed by this syntax:

eval <field> = <expression>
The following searches format all the string values in the categoryId field into lowercase, and then store the results in a new field called low_category:
Example scenarioSearch statement

Using the search command

 
search index=main | eval low_category = lower(categoryId)

Using the from command, starting with the SELECT clause

 
SELECT * FROM main | eval low_category = lower(categoryId)

Using the from command, starting with the FROM clause

 
FROM main | eval low_category = lower(categoryId)

Evaluate mathematical expressions

You can also evaluate mathematical expressions by using the eval command. For example, you can convert numerical values from bytes into kilobytes.

Note: For information about SPL2 expressions, see Types of expressions.

To add the eval command to your search, enter a pipe ( | ) character followed by this syntax:

eval <field> = <expression>
The following searches convert the values in the bytes field into kilobytes, round the converted values to 3 decimal places, and then store the results in a new field called kbytes:
Example scenarioSearch statement

Using the search command

 
search index=main | eval kbytes = round((bytes / 1024), 3)

Using the from command, starting with the SELECT clause

 
SELECT * FROM main | eval kbytes = round((bytes / 1024), 3)

Using the from command, starting with the FROM clause

 
FROM main | eval kbytes = round((bytes / 1024), 3)

Calculate a sum

You can calculate the sum of all the values in a field by using the stats command.

To add the stats command to your search, enter a pipe ( | ) character followed by this syntax:

stats <aggregation>
For example, the following searches calculate the sum of the values in the bytes field, and return a single field named sum(bytes) that contains the result:
Example scenarioSearch statement

Using the search command

 
search index=main | stats sum(bytes)

Using the from command, starting with the SELECT clause

 
SELECT * FROM main | stats sum(bytes)

Using the from command, starting with the FROM clause

 
FROM main | stats sum(bytes)

If you’re using the from command to search your data, then you can choose to specify this calculation in the optional SELECT clause of the command instead of adding a stats command to your search.

The following searches achieve the same results as the searches in the previous table:
Example scenarioSearch statement

Using the from command, starting with the SELECT clause

 
SELECT sum(bytes), _time, _raw FROM main

Using the from command, starting with the FROM clause

 
FROM main SELECT sum(bytes), _time, _raw
Note: The SELECT clauses in these examples include the _time and _raw fields in order to avoid dropping them from the search results. If you run a search in the Search page that drops the _time or _raw fields, the Timeline and Events viewer elements of the Search page do not display as expected.

Count the number of distinct values

You can return a count of the distinct values in a field by using the stats command.

To add the stats command to your search, enter a pipe ( | ) character followed by this syntax:

stats <aggregation>
For example, the following searches count the number of distinct values in the actions field for each client IP address. These searches return the clientip field and a field named dc(action) that contains the result.
Example scenarioSearch statement

Using the search command

 
search index=main | stats dc(action) by clientip

Using the from command, starting with the SELECT clause

 
SELECT * FROM main | stats dc(action) by clientip

Using the from command, starting with the FROM clause

 
FROM main | stats dc(action) by clientip

When using the from command to search your data, you can choose to specify this calculation in the optional SELECT and GROUP BY clauses of those commands instead of adding a stats command to your search.

The following searches achieve the same results as the searches in the previous table:
Example scenarioSearch statement

Using the from command, starting with the SELECT clause

 
SELECT dc(action), clientip, _time, _raw FROM main GROUP BY clientip

Using the from command, starting with the FROM clause

 
FROM main GROUP BY clientip SELECT dc(action), clientip, _time, _raw
Note: The SELECT clauses in these examples include the _time and _raw fields in order to avoid dropping them from the search results. If you run a search in the Search page that drops the _time or _raw fields, the Timeline and Events viewer elements of the Search page do not display as expected.

Next step

In many cases, when you use an SPL2 command or function to process your data, you create a new field to store the results of the command or function. After multiple data transformations, your events can become bloated with redundant or obsolete fields. Additionally, some fields might be generated with names that are not intuitive, or your processing actions might reframe the data in such a way that some pre-existing field names become removed from context.

For example, after you use an eval command to generate the kbytes field, the bytes field becomes redundant.

As another example, when you use the stats command to calculate an aggregation statistic, the name of the resulting field describes the aggregation function that was used, such as sum(bytes) or dc(action). These field names are not as meaningful or tied to context as names like total_website_traffic or action_categories.

You can sanitize your events by dropping unwanted fields or renaming fields.

To continue building out your search, see Modifying the fields in your search results.