Add-ons and CIM

Splunk add-ons are most commonly used to bring a new data source into the Splunk platform. Most add-on developers design their add-ons to be used with the Splunk Common Information Model (CIM) in order to work with the larger Splunk ecosystem. Splunk-developed add-ons provide the field extractions, lookups, and event types needed to map data to the CIM, allowing customers to easily use the new data source in data models, pivots, and CIM-based apps.

The Splunk Common Information Model add-on is not required to use add-on features such as data collection, prebuilt panels, or custom commands. You can use individual add-ons on their own, without installing the CIM add-on, if you do not want to map their data to the CIM.

To take advantage of the CIM mappings provided in an add-on, install the Splunk Common Information Model add-on to your search heads. The Splunk Common Information Model add-on is packaged with CIM-based apps such as Splunk Enterprise Security and the Splunk App for PCI Compliance. If you are using an add-on in conjunction with one of these apps, you do not need to install the Splunk Common Information Model add-on separately. However, the Splunk Common Information Model add-on is not packaged with all apps that are designed to use the CIM. In order to use data from an add-on with an app that relies on the CIM, you need to install the Splunk Common Information Model add-on.

A full list of apps that work with the Splunk Common Information Model is available on Splunkbase.

Source types for add-ons

All Splunk supported add-ons have one or more predefined source types to identify the type of data the add-on collects from the third-party system. Many source types support data models in the Common Information Model (CIM)

Source type naming conventions

Source type names use the following format:

vendor:product:technology:format

The shortest source type name is used to distinguish it from other source types. For example, if the vendor provides only a single format, then :format is not included in the source type name. If the vendor's product does not provide different log formats or sources for different technologies, then :technology is not included in the source type name. For example, the Splunk Add-on for OSSEC has a single source type called ossec. This source type name uses only one of the naming components because the add-on collects only one kind of data from the vendor, OSSEC.

Search using add-on source types

The source type naming format enables you to use wildcards at the end of a search term when you run a search to achieve the desired level of abstraction in the search results.

For example, you can run the following search to retrieve all Cisco logs.

Setting source type for input

Some Splunk add-ons have preconfigured inputs set to the appropriate source type for the third-party technology. During index time, the add-on separates the data into specific source types if there is more than one source type included with the add-on.

For add-ons that require you to create inputs to retrieve data from the third-party system, you must set the source type to a specific source type for the add-on technology as referenced in the documentation for the add-on. This source type tells the Splunk platform how to format the events during indexing. The CIM mappings and any dashboard panels provided with the add-on are also dependent on this source type. If the data inputs are not set to the correct source type, the CIM mappings and dashboard panels included with the add-on will not work.

For more information about source types, see Why source types matter in the Splunk Enterprise Getting Data In manual.

Add-ons and indexes

The Splunk platform stores the data that it collects in indexes. Understanding Splunk indexes is important for ensuring good performance when you search, for setting retention policies, and for providing data security (controlling who has access to the data). Out of the box, all data collected by Splunk supported add-ons is indexed to the default Splunk index, main. Splunk administrators are encouraged to change the index that is used for the source types in the add-on from the default index to another index that will meet the retention requirements and user access needs for this data source.

You can change the index that is used for the data source when configuring the add-on. Some add-ons include a setup page that allows you to specify the index to send your data to. Note that these setup pages can only list indexes that are locally configured on that node. For add-ons that do not include a setup page, or for hosts that cannot list the desired index, you can edit the inputs.conf file directly to specify the index to use for the data collected by the input. To do this, add the following line to the input's stanza in inputs.conf on the Splunk platform component where the data is entering the system, usually a forwarder:

index = <index_name>

Note that you must first create the index and ensure that it is in place on all nodes that may receive data before data from the add-on can be routed to it.

See Set up multiple indexes in the Managing Indexers and Clusters of Indexers manual for more information about creating and using indexes.

Selecting from multiple timestamps

Many Splunk add-ons collect event data from third-party products using syslog. You need to configure the third-party product to send syslog data to the Splunk platform in a format that the Splunk platform can understand. Configuration is slightly different for each event-producing or event-processing product.

It is very common practice in large enterprises to use multiple syslog systems to route and forward data before indexing in the Splunk platform. Depending on how these configuration choices are made, there can be multiple timestamps and/or hostnames included in an event when the Splunk platform sees it. If this is the case, we have no way of knowing which of the multiple timestamps or hostnames the customer wanted to use. Rather than ship unused extractions for all possible outcomes, the add-ons that expect syslog inputs have a single configuration. Customers may need to alter the regular expressions in these add-ons to extract the proper hostname and time stamp.

The Splunk platform normally looks at the text of the event for the timestamp, and by default will select the leftmost recognizable timestamp. Add-on specific configuration may have modified this behavior. Refer to the documentation for the individual add-on you are configuring. If there is an issue with using the timestamps included in the syslog events, you can modify props and transforms to select a different timestamp format.

Alternatively, you can change how the Splunk platform extracts timestamps. There may be cases where you would prefer to use the event's time of receipt, that is, the time the event is received to the Splunk platform, instead. To use the time of receipt as the event timestamp, add the setting DATETIME_CONFIG = NONE to the local props.conf for the add-on. For example, $SPLUNK_HOME/etc/apps/Splunk_TA_symantec-dlp/local/props.conf. This setting disables event-based timestamp processing, which means the Splunk platform will use the time the event is received at an indexer or heavy forwarder as the event timestamp. This can be preferred for multi-source syslog configurations with events from different time zones, because forcing a single relative time series makes correlation easier.

Selecting from multiple timezones

If an event timestamp does not include a recognizable timezone, the Splunk platform uses the time zone of the host that indexes the event. For example, if an event with field timestamp="2016-01-26 21:58:38.107" is indexed by a Splunk platform instance in GMT-8, it will subtract 8 hours from the timestamp and set the _time of event to: 2016-01-26T13:58:38.000-08:00. See How Splunk software determines time zones for details. The Splunk platform will also adjust the displayed _time based on the user's locale. See How browser locale affects timestamp formatting for more information.

It is not always possible to alter the logged time formats. Watch for potential issues at these points:

• Daylight savings time shifts. DST Fall Back events produce overlap in data streams. For example, if events are logging in EST (UTC-5) and shift to EDT (UTC-4), then new and old events will be interleaved. DST Spring Forward events produce gaps in data streams. For example, if events are logging in EST (UTC-5) and shift to EDT (UTC-4), then new events will be artificially offset from old events.
• Daylight savings time alterations. Daylight Savings is a legal construct rather than a physical one, and the rules change unpredictably with some regularity. These rule changes require OS and application level patches that are not reliably applied, which can cause systems in the same timezone to report with offsets from each other. Additionally, these offsets stop when patches are applied, which can be surprising.

Because this behavior can be confusing for data analysts in multiple timezones handling data sources from multiple timezones, Splunk recommends indexing data in UTC instead of local time. For information about timestamp configuration options, see Configure timestamp recognition in Getting Data In.

Add-ons and FIPS mode

FIPS mode is not currently supported by all Splunk-supported add-ons. Some add-ons with modular inputs use an encryption algorithm to store credentials. This encryption algorithm is not supported when FIPS is enabled on the Splunk platform. If you are required to use FIPS in your environment, the workaround for using this type of add-on is to install a heavy forwarder that is not using FIPS to perform data collection and then send that data to an indexer with FIPS enabled.

For more information about using FIPS with the Splunk platform, see Secure Splunk Enterprise with FIPS in the Securing Splunk Enterprise manual.

Support and resource links for add-ons

For general Splunk platform support, see Splunk Support Programs.

More resources

Access these Splunk platform resources for more help:

• Splunk Enterprise documentation
• Splunk Cloud documentation
• Splunk Light documentation
• Splunk Answers
• Splunk Community chat