Search using default fields

Whenever possible, specify the index, source, or source type in your search. These fields are added automatically to each event as default fields.

The key to fast searching is to limit the data that needs to be pulled from disk to an absolute minimum. Filter the data as early as possible in the search, so that processing is done on the minimum amount of data necessary.

Limit the data from disk

The techniques to limit the amount of data retrieved from disk range from setting a narrow time window, being as specific as possible, and retrieving the smallest number of events necessary.

Narrow the time window

One of the most effective ways to limit the data that is pulled off from disk is to limit the time range. Use the time range picker or specify time modifiers in your search to identify the smallest window of time necessary for your search.

If you need to see data from only the last hour, do not use the default time range of Last 24 hours.

If you must use a broad time range, such as Last week or All time, then use other techniques to limit the amount of data retrieved from disk.

Specify the index, source, or source type

Understanding how your data is organized is important to optimizing your searches. Take the time to learn which indexes contain your data, the sources of your data, and the source types. Knowing this information about your data helps you narrow down your searches.

  1. Run the following search.
    CODE 
    search source=*

    This search not optimized, but it does provide an opportunity for you to learn about the data you have access to.

  2. In the Fields list, click on each type of field and look at the values for host, source, and sourcetype.
  3. Click on the index field. Look at the names of the indexes that you have access to.

Whenever possible, specify the index, source, or source type in your search. When Splunk software indexes data, it automatically tags each event with a number of fields. The index, source, and source type fields are added automatically to each event as default fields.

Limit the number of events retrieved

By default, a Splunk search retrieves all events. However in some situations you might want to retrieve a sample set of events, instead of retrieving the entire event set. Limiting the number of events retrieved is useful in several situations:

  • You are creating a search and want to determine if you are retrieving the correct events.
  • You need only a subset or sample set of events for your search.

You can specify a limit to the number of events retrieved by using the head command. The head command retrieves only the most recent N events. For example:

CODE 
search  sourcetype=access_* | head 100 ...

Avoid using NOT expressions

More resources are used tracking NOT expressions than if you specify what you are looking for. Where ever possible, avoid using NOT expressions. For example, instead of using a string of NOT or != expressions such as:

from index=main where (NOT host=d OR host=e)

or

search host!=d OR host!=e

Use the specific terms you are searching for:

from index=main where (host=a OR host=b OR host=c).