Integrate the Log Auto-Discovery Extension

Note: Contact customer support for any questions or concerns regarding regional support or scale requirements.

The Log Auto-Discovery Extension enables you to use the Machine Agent with the Analytics Agent Extension to the Standalone Machine Agent to automatically detect and monitor log files. It is intended to support any Unix and Windows operating system (Windows support may vary). With this extension, you can:

  • Identify open log files of a process with a given string match, or files in a given directory matching a string.
  • Test a sample of the found log file to determine the file type.
  • Create a new job file for the log file in the Analytics Agent job directory. This job file will inform the Analytics Agent to begin monitoring based on those rules.
  • Automatically delete all generated job files when the Machine Agent shuts down. This allows for re-detection of application processes and associated log files, as well as updated log file message format matching.

With this solution, it is possible to perform real-time and automatic collection of logs through the Machine Agent.

Note: The monitored log files will not be displayed on the Controller. The logs and their data will be displayed in the Analytics log schema when you query SELECT * FROM logs. See Analytics Log Data and SELECT Clause.

Add the Extension to the Machine Agent

  1. Download the Log Auto-Discovery Extension zip file.
  2. Unzip the file and copy the LogMonitor directory to the directory <machine_agent_home>/monitors/ of your Machine Agent.Do not placethe extension in the extensions directory of your Machine Agent.
  3. Create a monitor.xml configuration file for the Machine Agent. The monitor.xml file enables the Machine Agent to use an extension. See Create the monitor.xml File.
  4. Configure the monitor.xml file with the path to the extension's configuration file default-config.yml. (Note that the path is relative to the path stored in the environment variable $AGENT_HOME.) 
    <task-arguments>
<argument name="config-file" is-required="true" default-value=“./default-config.yml” />
</task-arguments>
  5. Configure the extension with the file default-config.yml that lives in the root directory of the extension.The following YAML example shows you the structure and provides descriptions as comments for the properties. Replace the placeholder text with your own Controller information as well as the regex patterns for your key words to identify log file paths.
    #default configuration, this can be edited, and should be copied to a new file in case of upgrade
grokDirectory: ./grok
jobDirectory: ./jobs
#we will create the job files here: {analyticsDirectory}/conf/job
#this is optional, we will try and determine this automatically
#analyticsDirectory: ../analytics-agent
#operating system specific commands
psCommand: "/bin/ps -ef"
lsofCommand: "/usr/sbin/lsof -p %d"
#windows would have these:
#psCommand: pslist.exe
#lsofCommand: "handle.exe -p %d"
#to download pslist, https://download.sysinternals.com/files/PSTools.zip
#to download handle.exe, https://download.sysinternals.com/files/Handle.zip
process:
- java
- -Dappdynamics.com
logs:
- /var/log/*.log
#optionally, you can exclude log files where the absolutely path matches a given regex pattern:
#excludeLogs:
#  - .*appdynamics.*
    Note: Avoid using tab (\t) when editing YAML files. Splunk AppDynamics recommends using a YAML validator to validate the default-config.yml file.
  6. Restart the Machine Agent.

Use the Log Analytics Data

You can collect Log Analytics data from many types of log files, including instrumented and non-instrumented applications as well as infrastructure. Log Analytics can be used as a standalone component. You can search and analyze log data just as you do transaction data.

For more details about how to use the log data, see Using Analytics Data.