Use the CIM to normalize CPU performance metrics

This example illustrates how to normalize data for CIM-compliance for an IT Service Intelligence use case. This example provides two variations: one using Splunk Web, and another using configuration files from the command line.

Normalize data for CIM-compliance using Splunk Web

Step 1. Get your data in

For the purposes of this example, assume that you have already added data to your Splunk platform deployment. For instructions on adding data, see Getting Data In.

Step 2. Examine your data in context of the CIM

Make sure that the data that you want to extract has a dataset specified in the CIM. For example, if you want to build a KPI search based on a specific CPU performance metric, such as cpu_load_percent, review the Performance data model to make sure that the data model lists CPU as a dataset.

If the CIM does not contain the specific data that you want to extract for your KPI searches, you can use a Splunk add-on or apply the Common Information Model to your own data. See Design data models in the Splunk Enterprise Knowledge Manager Manual.

Step 3. Configure CIM-compliant event types

  1. From Splunk Web, select Settings > Data Models.
  2. Find the data model dataset that you want to map your data to, then identify its associated tags.
    For example, the CPU dataset in the Performance data model has the following tags associated with it:
    tag = performance[/topic/topic/body/section/ol/li/codeblock/div
       {""})  (div]tag = cpu
    3. [/topic/topic/body/section/ol/li {""}) Create an event type. [/topic/topic/body/section/ol/li/ol {""}) [/topic/topic/body/section/ol/li/ol/li {""}) Select [/topic/topic/body/section/ol/li/ol/li/b {""}) Settings > Event types (b]. (li][/topic/topic/body/section/ol/li/ol/li {""}) Click [/topic/topic/body/section/ol/li/ol/li/b {""}) New. (b] (li][/topic/topic/body/section/ol/li/ol/li {""}) In the [/topic/topic/body/section/ol/li/ol/li/b {""}) Add new (b] dialog, type the following values for the following fields. [/topic/topic/body/section/ol/li/ol/li/table {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/colspec {""}) (colspec][/topic/topic/body/section/ol/li/ol/li/table/tgroup/colspec {""}) (colspec][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Destination App: (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) ITSI (entry] (row][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Name: (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) Type the name of the event type. For example, [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/codeph {""}) cpu_metrics (codeph]. (entry] (row][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Search String: (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) Type a search string for the event type. For example, [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/codeph {""}) sourcetype=test_cpu_log (codeph]. (entry] (row][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Tag(s): (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) Type the tags associated with the data model dataset you are mapping to. For example, [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/codeph {""}) performance (codeph], [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/codeph {""}) cpu (codeph]. (entry] (row][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Color (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) Select a color for the event type. Priority determines which event type color displays for an event. For more information, see [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/xref {"unresolved-reference"}) ERROR - unresolved reference (SplunkCloud_9.3.2411_Knowledge_Abouteventtypepriorities) (xref]. (entry] (row][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/b {""}) Priority (b] (entry][/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry {""}) Select a priority from 1 to 10, with 1 being the highest and 10 being the lowest. For more information, see [/topic/topic/body/section/ol/li/ol/li/table/tgroup/tbody/row/entry/xref {"unresolved-reference"}) ERROR - unresolved reference (SplunkCloud_9.3.2411_Knowledge_Abouteventtypepriorities) (xref]. (entry] (row] (tbody] (tgroup] (table] (li] (ol] (li] [/topic/topic/body/section/ol/li {""}) Click [/topic/topic/body/section/ol/li/b {""}) Save (b]. (li]
[/topic/topic/body/section/p {""}) For more information, see [/topic/topic/body/section/p/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Defineeventtypes) (xref] in the Splunk Enterprise [/topic/topic/body/section/p/i {""}) Knowledge Manager Manual (i]. (p]
[/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 4. Verify your tags (title] [/topic/topic/body/section/p {""}) See [/topic/topic/body/section/p/xref {""}) Use the CIM to normalize data at search time (xref] for details. (p] (section] [/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 5. Make fields CIM-compliant (title] [/topic/topic/body/section/p {""}) Create field aliases to make fields CIM-compliant. (p] [/topic/topic/body/section/p {""}) [/topic/topic/body/section/p/b {""}) Note (b]: Field aliases do not support multi-value fields. For more information, see [/topic/topic/body/section/p/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Addaliasestofields) (xref]. (p] [/topic/topic/body/section/ol {""}) [/topic/topic/body/section/ol/li {""}) From Splunk Web, select [/topic/topic/body/section/ol/li/b {""}) Settings > Fields > Field Aliases (b]. (li] [/topic/topic/body/section/ol/li {""}) Click [/topic/topic/body/section/ol/li/b {""}) New (b]. (li] [/topic/topic/body/section/ol/li {""}) In the [/topic/topic/body/section/ol/li/b {""}) Add New (b] window, type the following: [/topic/topic/body/section/ol/li/ol {""}) [/topic/topic/body/section/ol/li/ol/li {""}) For [/topic/topic/body/section/ol/li/ol/li/b {""}) Destination App: (b], select ITSI. (li][/topic/topic/body/section/ol/li/ol/li {""}) For [/topic/topic/body/section/ol/li/ol/li/b {""}) Name: (b], type a name for your field alias. (li][/topic/topic/body/section/ol/li/ol/li {""}) For [/topic/topic/body/section/ol/li/ol/li/b {""}) Apply to: (b], select [/topic/topic/body/section/ol/li/ol/li/b {""}) Sourcetype (b]. (li][/topic/topic/body/section/ol/li/ol/li {""}) For [/topic/topic/body/section/ol/li/ol/li/b {""}) named: (b], type the name of the source type. For example, [/topic/topic/body/section/ol/li/ol/li/codeph {""}) test_cpu_log (codeph]. (li] (ol] (li] [/topic/topic/body/section/ol/li {""}) Restart the Splunk platform for your changes to take effect. (li] [/topic/topic/body/section/ol/li {""}) Create search-time field extractions. [/topic/topic/body/section/ol/li/div {""}) (div]If your event data contains fields that are not found in existing data models or search-time field extractions, you can add those fields using the Field Extractions page in Splunk Web. See [/topic/topic/body/section/ol/li/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Managesearch-timefieldextractions) (xref] in the [/topic/topic/body/section/ol/li/i {""}) Knowledge Manager Manual (i]. (li] [/topic/topic/body/section/ol/li {""}) Write lookups to add fields and normalize field values. (li] [/topic/topic/body/section/ol/li {""}) Verify fields and values. (li] (ol] (section] [/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 6. Validate normalized data against the data model (title] [/topic/topic/body/section/p {""}) Now that you have mapped your data to the CIM, you can validate that your data is CIM-compliant. See [/topic/topic/body/section/p/xref {""}) 6. Validate your data against the data model (xref]. (p] (section]

Normalize data for CIM-compliance using configuration files

This section demonstrates how to normalize data for CIM-compliance at search-time using Splunk configuration files.

Step 1. Get your data in

For the purposes of this example, assume that you have already added data to your Splunk platform deployment. For instructions on adding data, see Getting Data In.

Step 2. Examine your data in context of the CIM

Make sure that the data that you want to extract has a dataset specified in the CIM. For example, if you want to build a KPI search based on a specific CPU performance metric, such as cpu_load_percent, review the Performance data model to make sure that the data model lists CPU as a dataset.

If the CIM does not contain the specific data that you want to extract for your KPI searches, you can use a Splunk add-on or apply the Common Information Model to your own data. See Design data models in the Splunk Enterprise Knowledge Manager Manual.

Step 3. Configure CIM-compliant event tags

  1. Determine which tags are associated with the data model dataset. In Splunk Web, select Settings > Data Models.
  2. Find the data model dataset that you want to map your data to, then identify its associated tags.
    For example, the cpu_load_percent attribute in the CPU dataset in the Performance data model has the following tags associated with it:
    tag = performance[/topic/topic/body/section/ol/li/codeblock/div
       {""})  (div]tag = cpu
    3. [/topic/topic/body/section/ol/li {""}) On the search head, edit or create an [/topic/topic/body/section/ol/li/codeph {""}) $SPLUNK_HOME/etc/apps/$APPNAME$/local/eventtypes.conf (codeph] file, then manually add the event type. [/topic/topic/body/section/ol/li/div {""}) (div]For example: [/topic/topic/body/section/ol/li/codeblock {""}) [cpu_metrics] search = sourcetype=test_cpu_log (codeblock] (li] [/topic/topic/body/section/ol/li {""}) On the search head, edit or create a [/topic/topic/body/section/ol/li/codeph {""}) $SPLUNK_HOME/etc/apps/$APPNAME$/local/tags.conf (codeph] file, then manually add the appropriate tags for the data model dataset. For example:[/topic/topic/body/section/ol/li/div {""}) (div] [/topic/topic/body/section/ol/li/codeblock {""}) [eventtype=cpu_metrics] performance = enabled cpu = enabled (codeblock] (li] [/topic/topic/body/section/ol/li {""}) Restart the Splunk platform. (li]
[/topic/topic/body/section/p {""}) For more information, see [/topic/topic/body/section/p/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Configureeventtypes) (xref]. (p] [/topic/topic/body/section/p {""}) [/topic/topic/body/section/p/div {""}) (div] (p]
[/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 4. Verify your tags (title] [/topic/topic/body/section/p {""}) See [/topic/topic/body/section/p/xref {""}) Use the CIM to normalize data at search time (xref]. (p] (section] [/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 5. Make fields CIM-compliant (title] [/topic/topic/body/section/p {""}) Create field aliases to make fields CIM-compliant, then add search-time field extractions for additional fields as needed. (p] [/topic/topic/body/section/ol {""}) [/topic/topic/body/section/ol/li {""}) Create field aliases in [/topic/topic/body/section/ol/li/codeph {""}) props.conf (codeph]. You can create multiple field aliases in a single stanza. Create your field alias by adding the following line to a stanza in the [/topic/topic/body/section/ol/li/codeph {""}) $SPLUNK_HOME/etc/apps/$APPNAME$/local/props.conf (codeph] file. [/topic/topic/body/section/ol/li/div {""}) (div][/topic/topic/body/section/ol/li/codeblock {""}) FIELDALIAS-<class> = <orig_field_name> AS <new_field_name> (codeblock]For example:[/topic/topic/body/section/ol/li/div {""}) (div] [/topic/topic/body/section/ol/li/codeblock {""}) [test_cpu_log] FIELDALIAS-cpu_percent = cpu_percent AS cpu_load_percent (codeblock] (li] [/topic/topic/body/section/ol/li {""}) Restart the Splunk platform for your changes to take effect. (li] [/topic/topic/body/section/ol/li {""}) Create basic search-time field extractions in [/topic/topic/body/section/ol/li/codeph {""}) props.conf (codeph] by adding an EXTRACT stanza to [/topic/topic/body/section/ol/li/codeph {""}) $SPLUNK_HOME/etc/apps/$APPNAME$/local/props.conf (codeph]:[/topic/topic/body/section/ol/li/div {""}) (div][/topic/topic/body/section/ol/li/codeblock {""}) EXTRACT-<class> = [<regular_expression>|<regular_expression> in <source_field>] (codeblock] (li] (ol] [/topic/topic/body/section/p {""}) For more information about field aliases, see [/topic/topic/body/section/p/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Addaliasestofields) (xref] in the [/topic/topic/body/section/p/i {""}) Knowledge Manager Manual (i]. (p] [/topic/topic/body/section/p {""}) For more information about search-time field extractions, see [/topic/topic/body/section/p/xref {"unresolved-reference"}) ERROR - unresolved reference (Splunk_9.4.2_Knowledge_Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles__Create_basic_search-time_field_extractions_with_props.conf_edits) (xref]. (p] (section] [/topic/topic/body/section {""}) [/topic/topic/body/section/title {""}) Step 6. Validate normalized data against the data model (title] [/topic/topic/body/section/p {""}) Now that you have mapped your data to the CIM, you can validate that your data is CIM-compliant. See [/topic/topic/body/section/p/xref {""}) 6. Validate your data against the data model (xref]. (p] (section]