Starting Point for Creating a Source Rule

You can create a source rule from any one of these starting points:

  • Splunk AppDynamics template. Several templates for common log file formats are available.
  • Existing source rule. You can use existing source rules as the starting point for new rules.
  • New source rule. Start from scratch when your log file does not match one of the available templates.

Preview Extracted Log Data Using Sample Log Files

To improve validation of data collection and parsing of log messages, you can use a local log file with the Log Analytics Configuration UI to preview the field extraction that you want. Three types of field extraction are available.

  • Grok patterns and key-value pair extraction
  • Auto extraction using regular expressions
  • Manual extraction using regular expressions

Best Practices for Source Rule Design

A few recommendations apply to making source rules.

Use the simplest possible regular expressions and grok matching patterns possible. Do not make excessive use of wildcards or quantifiers as it can slow the responsiveness of the Controller UI. You can view examples of such greedy quantifiers on https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html.

If a regex pattern (including grok) is taking more than five seconds to match against a logline, extraction and further processing for those fields stop. If this occurs, some fields may be missing for that log line when viewed on the Controller. Other log lines are not impacted; however, this occurrence is often the result of an ineffective or faulty matching pattern in the first place, and processing is likely to take a long time for all log lines. This behavior is applicable to the dynamic Preview screens in the Centralized Log Analytics Configuration page as well.

The Controller limits the size of the records it retrieves to 32 KB by source rule in log analytics. This limit guards against excessive system resource burden, including potential resource burden that might be caused by excessive data collection due to faulty source rule patterns.