Migrate Log Analytics Job Files to Source Rules

The Centralized Log Management UI enables you to configure your log data sources using source rules. Manually created job files are no longer needed.

We recommend that you convert your log source configuration from job files (job files were used for configuration < 4.3) to source rules.

  1. Confirm that you have upgraded your Controller, Events Service, and Analytics Agent to 4.4. See these corresponding upgrade pages:
  2. Use the Centralized Log Management UI to create source rules for your existing job files by importing the configuration from your old job file to a new source rule. Be sure to use either 'Parsing time range' or 'End of log file'. If you do not use this, you will double collect all the log files that have already been tailed by the job file. Read Timing Notes for more information to help you decide on your collection timing settings. When you save the source rules, they are in the disabled state by default.
  3. Map the source rules to an Agent Scope.
  4. Enable the new source rules through the UI.
  5. Disable the job files that are actively collecting log analytics data. You do this by manually editing the job file and changing theenabled property to false. See Configure Log Analytics Using Job Files.
  6. After you have migrated all your job file configurations to source rules, you can completely disable the use of job files by clicking “Disable Field Extraction With Job Files”. This action can not be reversed and you will not be able to use job files after performing this action. Only trigger this action once you have completely moved to source rules created through the Centralized Log Management UI.
Timing Notes

By default, the controller communicates new configuration information to the Analytics Agent every five minutes. So it could be up to five minutes before the agent starts tailing your log file using the new source rule configuration. On the other hand, if you have both a job fileanda source rule enabled for the same log data, the data is collected twice. To avoid this situation, you can use configuration settings to specify when to start collecting the log data:

  • At the end of the file (UI field = Start collecting from End of log file)
  • During a specific time range (UI field = Parsing time range)

See the section describing the configuration settings on the "General Configuration Tab" in Configure Log Analytics Using Source Rules.

Be sure to disable the job file once you have enabled the source rule to collect the Log Analytics data. You can verify that the new source rule is collecting data correctly by waiting until the new log records appear in the Analytics Search UI data grid. One way to distinguish between the log data collected by a job file or a source rule is to use a different source type in the source rule.